tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab: QEMU running (pid 46) gitlab # Formatting '/build/vm-state-gitlab/tmp.vJvlemgPQZ', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) gitlab # Discarding device blocks: 0/262144 done gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: b7066820-71d2-454d-8666-3bded53d70c7 gitlab # Superblock backups stored on blocks: (finished: start all VMs, in 56.26 seconds) gitlab # 32768, 98304, 163840, 229376 gitlab: waiting for unit setup-gitlab.service gitlab # gitlab: waiting for the VM to finish booting gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/m0cdccz8nbrggsb7y80nxmivqm30vacw-nixos-system-gitlab-test/init regInfo=/nix/store/5nv3xnaj5zlx53wyvad679ya5y2pcp6i-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000000] kvm-clock: using sched offset of 474674108 cycles gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000005] tsc: Detected 2400.010 MHz processor gitlab # [ 0.000809] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000836] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000838] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000876] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002731] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002742] Using GB pages for direct mapping gitlab # [ 0.002900] RAMDISK: [mem 0xbe31c000-0xbffcffff] gitlab # [ 0.002907] ACPI: Early table checksum verification disabled gitlab # [ 0.002910] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002913] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002917] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002923] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002925] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.002927] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002928] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002930] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002931] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.002932] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.002933] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.002933] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.002934] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.002934] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003154] No NUMA configuration found gitlab # [ 0.003155] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003157] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff] gitlab # [ 0.005518] Zone ranges: gitlab # [ 0.005519] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.005520] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.005521] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005522] Device empty gitlab # [ 0.005523] Movable zone start for each node gitlab # [ 0.005523] Early memory node ranges gitlab # [ 0.005524] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.005525] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.005526] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005527] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.005548] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.005829] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.118463] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.118893] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.118908] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.118944] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.118947] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.118948] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.118950] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.118951] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.118951] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.118954] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.118955] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.118961] TSC deadline timer available gitlab # [ 0.118965] CPU topo: Max. logical packages: 1 gitlab # [ 0.118966] CPU topo: Max. logical dies: 1 gitlab # [ 0.118966] CPU topo: Max. dies per package: 1 gitlab # [ 0.118970] CPU topo: Max. threads per core: 1 gitlab # [ 0.118970] CPU topo: Num. cores per package: 4 gitlab # [ 0.118971] CPU topo: Num. threads per package: 4 gitlab # [ 0.118971] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.118995] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.119011] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.119014] kvm-guest: setup PV sched yield gitlab # [ 0.119031] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.119032] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.119033] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.119035] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.119036] Booting paravirtualized kernel on KVM gitlab # [ 0.119039] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.123482] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.125762] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.125815] kvm-guest: PV spinlocks enabled gitlab # [ 0.125817] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.125820] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/m0cdccz8nbrggsb7y80nxmivqm30vacw-nixos-system-gitlab-test/init regInfo=/nix/store/5nv3xnaj5zlx53wyvad679ya5y2pcp6i-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.125915] Unknown kernel command line parameters "regInfo=/nix/store/5nv3xnaj5zlx53wyvad679ya5y2pcp6i-closure-info/registration", will be passed to user space. gitlab # [ 0.125927] random: crng init done gitlab # [ 0.125928] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.135057] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.139787] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.139863] software IO TLB: area num 4. gitlab # [ 0.215933] Fallback order for Node 0: 0 gitlab # [ 0.215941] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.215943] Policy zone: Normal gitlab # [ 0.218537] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.228040] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.242278] allocated 12582912 bytes of page_ext gitlab # [ 0.252201] ftrace: allocating 48717 entries in 192 pages gitlab # [ 0.252203] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.253048] Dynamic Preempt: lazy gitlab # [ 0.253235] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.253236] rcu: RCU event tracing is enabled. gitlab # [ 0.253236] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.253238] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.253238] Rude variant of Tasks RCU enabled. gitlab # [ 0.253239] Tracing variant of Tasks RCU enabled. gitlab # [ 0.253239] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.253240] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.253261] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.253263] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.253264] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.257593] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.257927] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.257936] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.258045] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.261635] Console: colour VGA+ 80x25 gitlab # [ 0.261638] printk: legacy console [tty0] enabled gitlab # [ 0.292595] printk: legacy console [ttyS0] enabled gitlab # [ 0.407553] ACPI: Core revision 20250807 gitlab # [ 0.408560] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.410240] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.411353] x2apic enabled gitlab # [ 0.412138] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.413116] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.414469] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.416049] kvm-guest: setup PV IPIs gitlab # [ 0.417731] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.418877] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns gitlab # [ 0.420830] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010) gitlab # [ 0.421919] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.422946] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.424827] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.425832] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.426827] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.427828] Transient Scheduler Attacks: Vulnerable: No microcode gitlab # [ 0.428827] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.429827] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.430827] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.431835] Spectre V2 : Enabling IBPB for BPF gitlab # [ 0.432828] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.433828] active return thunk: srso_alias_return_thunk gitlab # [ 0.434851] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.436825] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.437827] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.438827] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.439826] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.441732] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.442827] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.443827] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.444827] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.446708] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.447826] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.448827] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.449827] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. gitlab # [ 0.479367] Freeing SMP alternatives memory: 44K gitlab # [ 0.479830] pid_max: default: 32768 minimum: 301 gitlab # [ 0.480940] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.481927] landlock: Up and running. gitlab # [ 0.482828] Yama: becoming mindful. gitlab # [ 0.484043] LSM support for eBPF active gitlab # [ 0.485047] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.486937] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.489602] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.490392] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.490832] ... version: 2 gitlab # [ 0.491613] ... bit width: 48 gitlab # [ 0.491829] ... generic counters: 6 gitlab # [ 0.492615] ... generic bitmap: 000000000000003f gitlab # [ 0.492883] ... fixed-purpose counters: 0 gitlab # [ 0.493679] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.494606] ... value mask: 0000ffffffffffff gitlab # [ 0.494828] ... max period: 00007fffffffffff gitlab # [ 0.495795] ... global_ctrl mask: 000000000000003f gitlab # [ 0.496730] signal: max sigframe size: 3376 gitlab # [ 0.497475] rcu: Hierarchical SRCU implementation. gitlab # [ 0.497832] rcu: Max phase no-delay instances is 400. gitlab # [ 0.498943] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.504322] smp: Bringing up secondary CPUs ... gitlab # [ 0.505271] smpboot: x86: Booting SMP configuration: gitlab # [ 0.505837] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.509963] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.511603] smpboot: Total of 4 processors activated (19200.08 BogoMIPS) gitlab # [ 0.512990] Memory: 6011596K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260108K reserved, 0K cma-reserved) gitlab # [ 0.514828] devtmpfs: initialized gitlab # [ 0.515920] x86/mm: Memory block size: 128MB gitlab # [ 0.518992] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.520917] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.521934] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.523172] PM: RTC time: 11:56:58, date: 2026-11-10 gitlab # [ 0.526991] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.529183] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.529884] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.531960] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.532842] audit: initializing netlink subsys (disabled) gitlab # [ 0.534049] audit: type=2000 audit(1794311818.388:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.534135] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.535829] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.536828] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.537828] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.538828] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.539843] cpuidle: using governor menu gitlab # [ 0.542084] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.543098] PCI: Using configuration type 1 for base access gitlab # [ 0.543830] PCI: Using configuration type 1 for extended access gitlab # [ 0.545035] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.547247] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.547830] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.548828] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.549828] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.554589] ACPI: Added _OSI(Module Device) gitlab # [ 0.556835] ACPI: Added _OSI(Processor Device) gitlab # [ 0.558838] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.565750] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.573475] ACPI: Interpreter enabled gitlab # [ 0.574946] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.576836] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.579067] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.582833] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.586312] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.598706] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.600841] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.604675] acpiphp: Slot [3] registered gitlab # [ 0.605891] acpiphp: Slot [4] registered gitlab # [ 0.607874] acpiphp: Slot [5] registered gitlab # [ 0.608871] acpiphp: Slot [6] registered gitlab # [ 0.609871] acpiphp: Slot [7] registered gitlab # [ 0.611889] acpiphp: Slot [8] registered gitlab # [ 0.612872] acpiphp: Slot [9] registered gitlab # [ 0.614871] acpiphp: Slot [10] registered gitlab # [ 0.615872] acpiphp: Slot [11] registered gitlab # [ 0.617893] acpiphp: Slot [12] registered gitlab # [ 0.618880] acpiphp: Slot [13] registered gitlab # [ 0.619871] acpiphp: Slot [14] registered gitlab # [ 0.621848] acpiphp: Slot [15] registered gitlab # [ 0.622652] acpiphp: Slot [16] registered gitlab # [ 0.623433] acpiphp: Slot [17] registered gitlab # [ 0.623848] acpiphp: Slot [18] registered gitlab # [ 0.624603] acpiphp: Slot [19] registered gitlab # [ 0.624849] acpiphp: Slot [20] registered gitlab # [ 0.625626] acpiphp: Slot [21] registered gitlab # [ 0.626849] acpiphp: Slot [22] registered gitlab # [ 0.627599] acpiphp: Slot [23] registered gitlab # [ 0.627849] acpiphp: Slot [24] registered gitlab # [ 0.628623] acpiphp: Slot [25] registered gitlab # [ 0.629849] acpiphp: Slot [26] registered gitlab # [ 0.630598] acpiphp: Slot [27] registered gitlab # [ 0.631872] acpiphp: Slot [28] registered gitlab # [ 0.632651] acpiphp: Slot [29] registered gitlab # [ 0.633411] acpiphp: Slot [30] registered gitlab # [ 0.633868] acpiphp: Slot [31] registered gitlab # [ 0.635037] PCI host bridge to bus 0000:00 gitlab # [ 0.635853] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.636829] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.637830] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.638830] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.640830] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.642861] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.646815] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.651158] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.655905] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.662907] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.664910] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.666833] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.668833] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.670832] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.673796] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.679891] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.683021] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.685100] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.685836] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.688064] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.711933] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.713852] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.714855] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.716340] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.718981] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.724867] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 0.725799] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 0.726674] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 0.738842] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 0.741373] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.745861] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 0.746801] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 0.747657] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 0.750101] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.756524] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 0.756854] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 0.758873] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 0.761052] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.763836] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 0.764743] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 0.765641] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 0.769078] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.772864] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 0.773813] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 0.774664] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 0.777353] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.781861] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 0.782935] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 0.784911] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 0.793140] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.803860] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 0.806851] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 0.808887] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 0.811851] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 0.818134] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.827073] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 0.829899] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 0.835969] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.844874] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 0.845845] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 0.848876] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 0.853177] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.857868] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 0.858837] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 0.859855] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 0.865756] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 0.866867] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 0.867970] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 0.869008] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 0.870921] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 0.873993] iommu: Default domain type: Translated gitlab # [ 0.874831] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 0.876157] ACPI: bus type USB registered gitlab # [ 0.877963] usbcore: registered new interface driver usbfs gitlab # [ 0.878850] usbcore: registered new interface driver hub gitlab # [ 0.879806] usbcore: registered new device driver usb gitlab # [ 0.882708] NetLabel: Initializing gitlab # [ 0.884710] NetLabel: domain hash size = 128 gitlab # [ 0.884710] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 0.886853] NetLabel: unlabeled traffic allowed by default gitlab # [ 0.887841] PCI: Using ACPI for IRQ routing gitlab # [ 0.889030] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 0.889825] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 0.889825] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 0.891833] vgaarb: loaded gitlab # [ 0.892700] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 0.893522] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 0.899035] clocksource: Switched to clocksource kvm-clock gitlab # [ 0.900685] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.901469] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.903121] pnp: PnP ACPI init gitlab # [ 0.904415] pnp: PnP ACPI: found 6 devices gitlab # [ 0.914287] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 0.915966] clocksource: Switched to clocksource acpi_pm gitlab # [ 0.917123] NET: Registered PF_INET protocol family gitlab # [ 0.919345] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.937115] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.938692] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.940186] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.944343] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.945782] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.947466] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.949164] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.950667] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.952103] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.953189] NET: Registered PF_XDP protocol family gitlab # [ 0.954116] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 0.955242] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 0.956324] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 0.957496] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.958694] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.960126] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 0.961176] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 0.963600] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 0.966202] PCI: CLS 0 bytes, default 64 gitlab # [ 0.967065] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 0.967714] Trying to unpack rootfs image as initramfs... gitlab # [ 0.968207] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB) gitlab # [ 0.968341] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns gitlab # [ 0.991822] Initialise system trusted keyrings gitlab # [ 0.993254] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 1.004709] Key type asymmetric registered gitlab # [ 1.005514] Asymmetric key parser 'x509' registered gitlab # [ 1.006462] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 1.008127] io scheduler mq-deadline registered gitlab # [ 1.009003] io scheduler kyber registered gitlab # [ 1.013709] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 1.015161] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 1.019230] Linux agpgart interface v0.103 gitlab # [ 1.021310] ACPI: bus type drm_connector registered gitlab # [ 1.026288] usbcore: registered new interface driver usbserial_generic gitlab # [ 1.029005] usbserial: USB Serial support registered for generic gitlab # [ 1.031517] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 1.031517] Please enable it if your BIOS has the CPPC option. gitlab # [ 1.037662] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 1.041533] drop_monitor: Initializing network drop monitor service gitlab # [ 1.044942] NET: Registered PF_INET6 protocol family gitlab # [ 1.049421] Segment Routing with IPv6 gitlab # [ 1.051447] In-situ OAM (IOAM) with IPv6 gitlab # [ 1.056068] IPI shorthand broadcast: enabled gitlab # [ 1.061968] sched_clock: Marking stable (908011277, 153888734)->(1089752535, -27852524) gitlab # [ 1.063968] registered taskstats version 1 gitlab # [ 1.065020] Loading compiled-in X.509 certificates gitlab # [ 1.075161] Demotion targets for Node 0: null gitlab # [ 1.076291] Key type .fscrypt registered gitlab # [ 1.077014] Key type fscrypt-provisioning registered gitlab # [ 1.078046] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 1.079055] ima: Allocated hash algorithm: sha1 gitlab # [ 1.079950] ima: No architecture policies found gitlab # [ 1.081117] PM: Magic number: 2:838:933 gitlab # [ 1.082801] RAS: Correctable Errors collector initialized. gitlab # [ 1.087364] clk: Disabling unused clocks gitlab # [ 1.088120] PM: genpd: Disabling unused power domains gitlab # [ 1.267954] Freeing initrd memory: 29392K gitlab # [ 1.275402] Freeing unused decrypted memory: 2028K gitlab # [ 1.281945] Freeing unused kernel image (initmem) memory: 3640K gitlab # [ 1.284628] Write protecting the kernel read-only data: 32768k gitlab # [ 1.289635] Freeing unused kernel image (text/rodata gap) memory: 1220K gitlab # [ 1.293657] Freeing unused kernel image (rodata/data gap) memory: 760K gitlab # [ 1.363728] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.364879] Run /init as init process gitlab # [ 1.378454] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.421573] fuse: init (API version 7.45) gitlab # [ 1.429700] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 1.436919] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 1.440343] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 1.476002] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 1.812552] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.853697] systemd[1]: Detected virtualization kvm. gitlab # [ 1.856027] systemd[1]: Detected architecture x86-64. gitlab # [ 1.858203] systemd[1]: Running in initrd. gitlab # [ 1.860759] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.863830] systemd[1]: Hostname set to . gitlab # [ 1.974415] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 2.020146] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 2.031550] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 2.032785] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 2.034281] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 2.035489] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 2.036651] systemd[1]: Reached target Path Units. gitlab # [ 2.037588] systemd[1]: Reached target Slice Units. gitlab # [ 2.038529] systemd[1]: Reached target Swaps. gitlab # [ 2.039358] systemd[1]: Reached target Timer Units. gitlab # [ 2.040387] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 2.041648] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 2.042851] systemd[1]: Listening on Journal Sockets. gitlab # [ 2.043940] systemd[1]: Listening on udev Control Socket. gitlab # [ 2.045025] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 2.046039] systemd[1]: Reached target Socket Units. gitlab # [ 2.048112] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 2.050321] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 2.052454] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 2.063109] systemd[1]: Starting Journal Service... gitlab # [ 2.067740] systemd[1]: Starting Load Kernel Modules... gitlab # [ 2.073100] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 2.079064] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 2.081371] systemd-journald[91]: Collecting audit messages is disabled. gitlab # [ 2.092078] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 2.097203] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.103868] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.107781] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.119315] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.121687] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.122514] netfs: FS-Cache loaded gitlab # [ 2.129303] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 2.129834] 9pnet: Installing 9P2000 support gitlab # [ 2.133330] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.136507] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.138295] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.140116] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.143031] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 1.990641] systemd-modules-load[93]: Using 4[ 2.144839] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # probe threads gitlab # [ 2.146450] systemd[1]: Started Journal Service. gitlab # [ 1.993426] systemd-modules-load[93]: Inserted module 'virtio_balloon' gitlab # [ 1.994720] systemd-modules-load[93]: Inserted module 'virtio_gpu' gitlab # [ 1.995744] systemd-modules-load[93]: Inserted module 'dm_mod' gitlab # [ 2.003167] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 2.004287] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 2.005650] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 2.006658] systemd[1]: Reached target Local File Systems. gitlab # [ 2.007408] systemd[1]: Starting Create System Files and Directories... gitlab # [ 2.008328] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 2.015490] systemd[1]: Finished Create System Files and Directories. gitlab # [ 2.031116] systemd-udevd[112]: Using default interface naming scheme 'v261'. gitlab # [ 2.041829] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 2.097061] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.097994] systemd[1]: Reached target System Initialization. gitlab # [ 2.098885] systemd[1]: Reached target Basic System. gitlab # [ 2.343727] SCSI subsystem initialized gitlab # [ 2.361796] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 2.363660] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 2.364882] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 2.366966] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 2.368240] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 2.370955] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 2.372155] usb usb1: Product: UHCI Host Controller gitlab # [ 2.373135] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd gitlab # [ 2.374492] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 2.379093] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 2.380375] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 2.382651] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 2.387190] hub 1-0:1.0: USB hub found gitlab # [ 2.387282] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 2.387521] hub 1-0:1.0: 2 ports detected gitlab # [ 2.387674] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 2.388844] scsi host0: ata_piix gitlab # [ 2.396055] scsi host1: ata_piix gitlab # [ 2.396853] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 2.396872] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 2.399395] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.401724] virtio_blk virtio5: [vdb] 11953392 512-byte logical blocks (6.12 GB/5.70 GiB) gitlab # [ 2.270072] (udev-worker)[135]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.272571] (udev-worker)[116]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.274841] (udev-worker)[116]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.282460] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.314046] systemd-vconsole-setup[151]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.315754] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.556056] ata2: found unknown device (class 0) gitlab # [ 2.557662] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 2.560282] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 2.593068] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # [ 2.606220] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 2.456895] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.462152] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.464250] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.466196] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.625814] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 2.503944] systemd-fsck[157]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.662602] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 2.511645] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.774100] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 2.777182] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 2.780602] usb 1-1: Product: QEMU USB Tablet gitlab # [ 2.782461] usb 1-1: Manufacturer: QEMU gitlab # [ 2.784040] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 2.814730] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 2.827656] usbcore: registered new interface driver usbhid gitlab # [ 2.829851] usbhid: USB HID core driver gitlab # [ 2.839640] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 2.843167] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 2.935956] systemd[1]: Mounting /sysroot... gitlab # [ 3.249626] EXT4-fs (vda): mounted filesystem b7066820-71d2-454d-8666-3bded53d70c7 r/w with ordered data mode. Quota mode: none. gitlab # [ 3.103122] systemd[1]: Mounted /sysroot. gitlab # [ 3.105059] systemd[1]: Reached target Initrd Root File System. gitlab # [ 3.109262] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 3.111296] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 3.115297] systemd[1]: Mounting /sysroot/run... gitlab # [ 3.117124] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.121418] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.125647] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.137680] systemd[1]: Mounted /sysroot/run. gitlab # [ 3.139718] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.146061] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.301324] 9p: Installing v9fs 9p2000 file system support gitlab # [ 3.148688] systemd-sysroot-fstab-check[177]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.304253] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.153188] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.154733] systemd[1]: Reload requested from client PID 177 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.156500] systemd[1]: Reloading... gitlab # [ 3.214244] systemd[1]: Reloading finished in 59 ms. gitlab # [ 3.234636] systemd-sysroot-fstab-check[177]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.237579] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 3.239342] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.241172] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.243667] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.245748] systemd-sysroot-fstab-check[177]: Requesting swap.target/start/replace... gitlab # [ 3.248295] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.250398] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.252778] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.255108] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.269586] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.271971] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.936730] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 3.990701] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 3.993926] systemd[1]: Reached target Initrd File Systems. gitlab # [ 3.998137] systemd[1]: Starting Find NixOS closure... gitlab # [ 4.001963] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 4.026921] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 4.030442] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 4.049520] systemd[1]: Finished Find NixOS closure. gitlab # [ 4.051771] systemd[1]: Reached target Initrd Default Target. gitlab # [ 4.053975] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 4.083712] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 4.086075] systemd[1]: Stopped target Basic System. gitlab # [ 4.087901] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 4.089806] systemd[1]: Stopped target Path Units. gitlab # [ 4.091543] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.094377] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.098417] systemd[1]: Stopped target Slice Units. gitlab # [ 4.100556] systemd[1]: Stopped target Socket Units. gitlab # [ 4.102480] systemd[1]: Stopped target System Initialization. gitlab # [ 4.105917] systemd[1]: Stopped target Swaps. gitlab # [ 4.108205] systemd[1]: Stopped target Timer Units. gitlab # [ 4.110663] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.112920] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.115830] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.118586] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.120431] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 4.124310] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.126223] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.128351] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.129834] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.131749] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.133020] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.134985] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.136919] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.138734] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.140269] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.141687] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.143374] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.144983] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.145822] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.146858] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.147947] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.148694] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.149652] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.150516] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.151487] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.152328] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.153239] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.154226] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.155173] systemd[1]: Closed udev Control Socket. gitlab # [ 4.155864] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.156718] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.157789] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.158733] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.160036] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.161107] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.162113] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.182969] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.184134] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.184988] systemd[1]: Reached target Switch Root. gitlab # [ 4.187376] systemd[1]: Starting NixOS Activation... gitlab # [ 4.323611] initrd-nixos-activation-start[229]: booting system configuration /nix/store/m0cdccz8nbrggsb7y80nxmivqm30vacw-nixos-system-gitlab-test gitlab # [ 4.369124] initrd-nixos-activation-start[229]: running activation script... gitlab # [ 4.598709] initrd-nixos-activation-start[252]: setting up /etc... gitlab # [ 4.709983] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.712800] systemd[1]: Finished NixOS Activation. gitlab # [ 4.714514] systemd[1]: Starting Switch Root... gitlab # [ 4.748817] systemd[1]: Switching root. gitlab # [ 4.966751] systemd-journald[91]: Received SIGTERM from PID 1 (systemd). gitlab # [ 5.089427] NET: Registered PF_VSOCK protocol family gitlab # [ 5.452305] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.462578] systemd[1]: Detected virtualization kvm. gitlab # [ 5.464476] systemd[1]: Detected architecture x86-64. gitlab # [ 5.466485] systemd[1]: Detected first boot. gitlab # [ 5.469104] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.609373] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.708676] systemd[1]: Applying preset policy. gitlab # [ 5.847441] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 5.968042] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 5.969448] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 5.971778] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 5.973862] systemd[1]: Created slice Slice /system/getty. gitlab # [ 5.975275] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 5.976782] systemd[1]: Created slice User and Session Slice. gitlab # [ 5.977729] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 5.979026] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 5.980130] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 5.980859] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 5.981669] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 5.982557] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 5.983360] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 5.984190] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 5.985056] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 5.986024] systemd[1]: Reached target Path Units. gitlab # [ 5.986760] systemd[1]: Reached target Remote File Systems. gitlab # [ 5.987660] systemd[1]: Reached target Slice Units. gitlab # [ 5.988387] systemd[1]: Reached target Swaps. gitlab # [ 5.989816] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 5.991820] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 5.993279] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 5.994817] systemd[1]: Listening on Factory Reset Management. gitlab # [ 5.995825] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 5.998223] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 5.999708] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 6.001569] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 6.002748] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 6.003969] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.005359] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 6.008314] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 6.009379] systemd[1]: Listening on udev Control Socket. gitlab # [ 6.010294] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 6.013081] systemd[1]: Mounting Huge Pages File System... gitlab # [ 6.015306] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 6.017308] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 6.021096] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 6.028198] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 6.029321] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 6.033446] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 6.035811] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 6.039757] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.045648] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.081574] systemd[1]: Mounting FUSE Control File System... gitlab # [ 6.082552] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.086687] systemd[1]: Starting Journal Service... gitlab # [ 6.088796] systemd[1]: Starting Load Kernel Modules... gitlab # [ 6.097062] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 6.102289] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 6.104971] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.108814] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 6.112160] systemd-journald[324]: Collecting audit messages is enabled. gitlab # [ 6.114749] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 6.115758] systemd[1]: Mounted Huge Pages File System. gitlab # [ 6.116726] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 6.121134] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 6.122110] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 6.122915] EXT4-fs (vda): re-mounted b7066820-71d2-454d-8666-3bded53d70c7. gitlab # [ 6.126050] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 6.126470] loop: module loaded gitlab # [ 6.127307] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 6.129069] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 5.976204] systemd[1]: Queue[ 6.130124] systemd[1]: Started Journal Service. gitlab # d start job for default target Multi-User System. gitlab # [ 5.978074] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 5.979238] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 5.980485] systemd-modules-load[325]: Using 3 probe threads gitlab # [ 5.981538] systemd-modules-load[325]: Inserted module 'tls' gitlab # [ 5.982495] systemd-modules-load[325]: Inserted module 'loop' gitlab # [ 5.983617] systemd[1]: Mounted FUSE Control File System. gitlab # [ 5.984567] systemd[1]: Finished Load Kernel Modules. gitlab # [ 5.985512] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 5.986591] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 5.987723] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 5.989064] systemd-oomd[327]: No swap; memory pressure usage will be degraded gitlab # [ 5.990479] systemd[1]: Starting Firewall... gitlab # [ 5.996759] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 5.998027] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.004069] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 6.005044] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 6.162841] systemd-journald[324]: Received client request to flush runtime journal. gitlab # [ 6.052089] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 6.054265] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.056523] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 6.058181] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 6.059448] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 6.060323] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 6.061185] systemd[1]: Reached target First Boot Complete. gitlab # [ 6.062039] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 6.063111] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 6.064148] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 6.065162] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 6.066093] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 6.067278] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 6.095033] systemd-udevd[373]: Using default interface naming scheme 'v261'. gitlab # [ 6.102912] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 6.129630] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.211922] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.237047] systemd[1]: Finished Firewall. gitlab # [ 6.251275] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 6.259594] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 6.276988] (udev-worker)[447]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.281270] (udev-worker)[428]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 6.283315] (udev-worker)[428]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.446177] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 6.448096] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 6.449744] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 6.454342] ACPI: button: Power Button [PWRF] gitlab # [ 6.304684] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 6.319499] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.321070] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.323302] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.324858] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.326334] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.497697] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 6.500686] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 6.506134] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 6.513650] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 6.513749] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 6.514156] FDC 0 is a S82078B gitlab # [ 6.519636] Console: switching to colour dummy device 80x25 gitlab # [ 6.520970] rtc_cmos 00:05: registered as rtc0 gitlab # [ 6.521293] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 6.521576] rtc_cmos 00:05: setting system clock to 2026-11-10T11:57:04 UTC (1794311824) gitlab # [ 6.522199] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 6.525388] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 6.525525] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 6.527063] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 6.539114] Console: switching to colour frame buffer device 160x50 gitlab # [ 6.541838] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 6.545012] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6 gitlab # [ 6.546920] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 6.561596] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 gitlab # [ 6.493142] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.662704] ppdev: user-space parallel port driver gitlab # [ 6.512141] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.513457] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.517346] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.674457] mousedev: PS/2 mouse device common for all mice gitlab # [ 6.522810] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.524168] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.534430] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.706307] kvm_amd: TSC scaling supported gitlab # [ 6.706820] kvm_amd: Nested Virtualization enabled gitlab # [ 6.707335] kvm_amd: Nested Paging enabled gitlab # [ 6.707769] kvm_amd: LBR virtualization supported gitlab # [ 6.708391] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 6.709064] kvm_amd: Virtual GIF supported gitlab # [ 6.709568] kvm_amd: Virtual NMI enabled gitlab # [ 6.743178] EDAC MC: Ver: 3.0.0 gitlab # [ 6.610597] systemd-vconsole-setup[495]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 6.613605] systemd[1]: Finished Virtual Console Setup. gitlab # [ 6.819476] systemd[1]: Mounting /run/wrappers... gitlab # [ 6.842233] systemd[1]: Mounted /run/wrappers. gitlab # [ 6.845164] systemd[1]: Reached target Local File Systems. gitlab # [ 6.848517] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 6.853124] systemd[1]: Starting register-nix-paths.service... gitlab # [ 6.857521] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 6.859453] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.868580] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 6.874139] systemd[1]: Starting Create System Files and Directories... gitlab # [ 6.913108] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 6.918101] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 6.922677] systemd[1]: Finished Create System Files and Directories. gitlab # [ 6.929200] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 6.932518] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 6.955905] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 6.965769] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 6.967145] systemd[1]: Starting Update is Completed... gitlab # [ 6.983454] systemd[1]: Finished Update is Completed. gitlab # [ 7.031871] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 7.033983] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.258074] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.260089] systemd[1]: Reached target System Initialization. gitlab # [ 7.261935] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.264150] systemd[1]: Started logrotate.timer. gitlab # [ 7.265631] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.267681] systemd[1]: Reached target Timer Units. gitlab # [ 7.269189] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.271193] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.272844] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.274621] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.278297] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.281473] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.283546] systemd[1]: Reached target Socket Units. gitlab # [ 7.316449] dbus-broker-launch[577]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.319668] dbus-broker-launch[577]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.322311] dbus-broker-launch[577]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.327540] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.343546] dbus-broker-launch[577]: Ready gitlab # [ 7.348155] systemd[1]: Reached target Basic System. gitlab # [ 7.351379] systemd[1]: Started backdoor.service. gitlab # [ 7.354298] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.364303] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.366982] systemd[1]: Starting Logrotate configuration check... gitlab # [ 7.373274] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # connecting to host... gitlab # [ 7.385250] systemd[1]: Starting Post-Boot Actions... gitlab # [ 7.392254] systemd[1]: Started Reset console on configuration changes. gitlab # [ 7.396277] logrotate[582]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # [ 7.399120] logrotate[582]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf gitlab # [ 7.400264] systemd[1]: Starting resolvconf update... gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 7.406544] logrotate[582]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 7.409565] logrotate[582]: Reading state from file: /var/lib/logrotate.status gitlab: connected to guest root shell gitlab # [ 7.411678] systemd[1]: Finished Post-Boot Actions. gitlab: (connecting took 8.04 seconds) gitlab: (finished: waiting for the VM to finish booting, in 8.04 seconds) gitlab # [ 7.414426] logrotate[582]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.418286] logrotate[582]: Allocating hash table for state file, size 64 entries gitlab # [ 7.421130] logrotate[582]: Handling 4 logs gitlab # [ 7.421786] logrotate[582]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.423982] logrotate[582]: considering log /var/log/btmp gitlab # [ 7.426245] logrotate[582]: Creating new state gitlab # [ 7.426935] logrotate[582]: Now: 2026-11-10 11:57 gitlab # [ 7.427680] logrotate[582]: Last rotated at 2026-11-10 11:00 gitlab # [ 7.428630] logrotate[582]: log does not need rotating (log has already been rotated) gitlab # [ 7.430179] logrotate[582]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.432326] logrotate[582]: considering log /var/log/wtmp gitlab # [ 7.433095] logrotate[582]: Creating new state gitlab # [ 7.433792] logrotate[582]: Now: 2026-11-10 11:57 gitlab # [ 7.434637] logrotate[582]: Last rotated at 2026-11-10 11:00 gitlab # [ 7.435609] logrotate[582]: log does not need rotating (log has already been rotated) gitlab # [ 7.436864] logrotate[582]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 7.438905] logrotate[582]: switching euid from 0 to 165 and egid from 0 to 165 (pid 582) gitlab # [ 7.440403] logrotate[582]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.440683] logrotate[582]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.441192] logrotate[582]: Creating new state gitlab # [ 7.441708] logrotate[582]: switching euid from 165 to 0 and egid from 165 to 0 (pid 582) gitlab # [ 7.442381] logrotate[582]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.442544] logrotate[582]: switching euid from 0 to 60 and egid from 0 to 60 (pid 582) gitlab # [ 7.446075] logrotate[582]: considering log /var/log/nginx/*.log gitlab # [ 7.446580] logrotate[582]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.447102] logrotate[582]: Creating new state gitlab # [ 7.447577] logrotate[582]: not running postrotate script, since no logs were rotated gitlab # [ 7.447974] logrotate[582]: switching euid from 60 to 0 and egid from 60 to 0 (pid 582) gitlab # [ 7.449693] nsncd[583]: Nov 10 11:57:05.560 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.450306] systemd[1]: Finished Logrotate configuration check. gitlab # [ 7.451316] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 7.451906] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.452391] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.452923] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.453935] systemd[1]: Starting User Login Management... gitlab # [ 7.478106] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.478748] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.479977] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.480266] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.480844] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 7.484206] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 7.484612] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 7.485512] systemd[1]: nscd.service: Consumed 23ms CPU time over 111ms wall clock time, 7.4M memory peak, 5.5M read from disk. gitlab # [ 7.489221] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.492767] systemd-logind[622]: New seat seat0. gitlab # [ 7.494909] systemd-logind[622]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 7.495359] systemd-logind[622]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 7.496802] systemd-logind[622]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 7.497547] systemd[1]: Started User Login Management. gitlab # [ 7.501393] systemd[1]: Starting linger-users.service... gitlab # [ 7.508326] systemd[1]: Finished resolvconf update. gitlab # [ 7.510630] systemd[1]: Reached target Preparation for Network. gitlab # [ 7.511887] nsncd[676]: Nov 10 11:57:05.643 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.515178] systemd[1]: Starting DHCP Client... gitlab # [ 7.518255] systemd[1]: Starting Address configuration of eth1... gitlab # [ 7.519416] systemd[1]: Starting Extra networking commands.... gitlab # [ 7.519901] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.520530] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 7.521325] systemd[1]: Finished linger-users.service. gitlab # [ 7.525670] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.526047] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.542712] network-addresses-eth1-start[699]: adding address 192.168.1.1/24... done gitlab # [ 7.545321] network-addresses-eth1-start[699]: adding address 2001:db8:1::1/64... done gitlab # [ 7.551297] systemd[1]: Finished Address configuration of eth1. gitlab # [ 7.582703] systemd[1]: Finished Extra networking commands.. gitlab # [ 7.585635] systemd[1]: Reached target Network. gitlab # [ 7.585948] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 7.587311] systemd[1]: Starting Nginx Web Server... gitlab # [ 7.591153] systemd[1]: Starting PostgreSQL Server... gitlab # [ 7.597101] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 7.598921] systemd[1]: Starting Permit User Sessions... gitlab # [ 7.603779] dhcpcd[728]: dhcpcd-10.3.2 starting gitlab # [ 7.606074] dhcpcd[770]: dev: loaded udev gitlab # [ 7.769215] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 7.769779] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 7.625803] systemd[1]: Finished Permit User Sessions. gitlab # [ 7.636375] systemd[1]: Started Getty on tty1. gitlab # [ 7.636775] systemd[1]: Reached target Login Prompts. gitlab # [ 7.644777] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 7.863283] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 7.868322] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 7.869226] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 7.871292] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 7.872506] cfg80211: failed to load regulatory.db gitlab # [ 7.730166] nginx-pre-start[788]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok gitlab # [ 7.732594] nginx-pre-start[788]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful gitlab # [ 7.737183] systemd[1]: Started Nginx Web Server. gitlab # [ 7.906609] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 7.754111] redis-server[789]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 7.757427] redis-server[789]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 7.759747] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 7.760875] redis-server[789]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=789, just started gitlab # [ 7.763052] redis-server[789]: Configuration loaded gitlab # [ 7.764273] redis[789]: monotonic clock: POSIX clock_gettime gitlab # [ 7.766538] postgresql-pre-start[792]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 7.768332] postgresql-pre-start[792]: This user must also own the server process. gitlab # [ 7.770249] dhcpcd[770]: eth0: waiting for carrier gitlab # [ 7.771219] postgresql-pre-start[792]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 7.772634] postgresql-pre-start[792]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 7.774076] postgresql-pre-start[792]: The default text search configuration will be set to "english". gitlab # [ 7.775581] postgresql-pre-start[792]: Data page checksums are enabled. gitlab # [ 7.776884] postgresql-pre-start[792]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 7.778520] postgresql-pre-start[792]: creating subdirectories ... ok gitlab # [ 7.779598] postgresql-pre-start[792]: selecting dynamic shared memory implementation ... posix gitlab # [ 7.781034] dhcpcd[770]: eth0: carrier acquired gitlab # [ 7.781899] redis[789]: Running mode=standalone, port=0. gitlab # [ 7.783283] redis[789]: Server initialized gitlab # [ 7.784229] redis[789]: Ready to accept connections unix gitlab # [ 7.785323] redis[789]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 7.787309] dhcpcd[770]: DUID 00:01:00:01:32:85:c3:11:52:54:00:12:34:56 gitlab # [ 7.788957] dhcpcd[770]: eth0: IAID 00:12:34:56 gitlab # [ 7.789849] dhcpcd[770]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 7.803746] postgresql-pre-start[792]: selecting default "max_connections" ... 100 gitlab # [ 7.846225] postgresql-pre-start[792]: selecting default "shared_buffers" ... 128MB gitlab # [ 8.177244] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.258978] postgresql-pre-start[792]: selecting default time zone ... UTC gitlab # [ 8.260610] postgresql-pre-start[792]: creating configuration files ... ok gitlab # [ 8.368567] gitlab-workhorse[825]: time="2026-11-10T11:57:06Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 8.368805] gitlab-workhorse[825]: time="2026-11-10T11:57:06Z" level=info msg="Using redis/go-redis" gitlab # [ 8.369682] gitlab-workhorse[825]: time="2026-11-10T11:57:06Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 8.429416] postgresql-pre-start[792]: running bootstrap script ... ok gitlab # [ 8.833223] postgresql-pre-start[792]: performing post-bootstrap initialization ... ok gitlab # [ 9.009439] dhcpcd[770]: eth0: soliciting a DHCP lease gitlab # [ 9.169673] NET: Registered PF_PACKET protocol family gitlab # [ 9.023918] dhcpcd[770]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 9.029446] dhcpcd[770]: eth0: probing address 10.0.2.15/24 gitlab # [ 9.155867] dhcpcd[770]: eth0: soliciting an IPv6 router gitlab # [ 9.156780] dhcpcd[770]: eth0: Router Advertisement from fe80::2 gitlab # [ 9.158376] dhcpcd[770]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 9.160170] dhcpcd[770]: eth0: adding route to fec0::/64 gitlab # [ 9.161432] dhcpcd[770]: eth0: adding default route via fe80::2 gitlab # [ 9.166532] postgresql-pre-start[792]: syncing data to disk ... ok gitlab # [ 9.167185] postgresql-pre-start[792]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 9.168354] postgresql-pre-start[792]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 9.169933] postgresql-pre-start[792]: Success. You can now start the database server using: gitlab # [ 9.171502] postgresql-pre-start[792]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 9.296284] postgres[850]: [850] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 9.300998] postgres[850]: [850] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 9.301150] postgres[850]: [850] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 9.311294] postgres[850]: [850] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 9.328925] postgres[860]: [860] LOG: database system was shut down at 2026-11-10 11:57:06 GMT gitlab # [ 9.341480] postgres[850]: [850] LOG: database system is ready to accept connections gitlab # [ 9.352076] systemd[1]: Started PostgreSQL Server. gitlab # [ 9.352935] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 9.574870] postgresql-setup-start[871]: CREATE DATABASE gitlab # [ 9.605524] postgresql-setup-start[890]: CREATE ROLE gitlab # [ 9.622960] postgresql-setup-start[892]: ALTER DATABASE gitlab # [ 9.648336] postgresql-setup-start[897]: CREATE ROLE gitlab # [ 9.651401] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 9.651973] systemd[1]: Reached target PostgreSQL. gitlab # [ 9.653949] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 9.687059] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[880]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 9.692532] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[880]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 9.740572] systemd[1]: Finished gitlab-config.service. gitlab # [ 9.747281] systemd[1]: Started gitaly.service. gitlab # [ 9.748842] gitlab-postgresql-start[906]: CREATE DATABASE gitlab # [ 9.753136] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 9.787985] gitlab-postgresql-start[923]: CREATE EXTENSION gitlab # [ 9.878300] gitlab-postgresql-start[927]: CREATE EXTENSION gitlab # [ 9.883704] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 10.284566] gitaly[919]: time="2026-11-10T11:57:08.415Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=919 gitlab # [ 10.284947] gitaly[919]: time="2026-11-10T11:57:08.416Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=919 gitlab # [ 10.286130] gitaly[919]: time="2026-11-10T11:57:08.416Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=919 gitlab # [ 10.286956] gitaly[919]: time="2026-11-10T11:57:08.416Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=919 gitlab # [ 10.288616] gitaly[919]: time="2026-11-10T11:57:08.416Z" level=info msg="Starting Gitaly" pid=919 version= gitlab # [ 10.468319] gitaly[919]: time="2026-11-10T11:57:08.600Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=171 pid=919 gitlab # [ 10.468615] gitaly[919]: time="2026-11-10T11:57:08.600Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=919 gitlab # [ 10.469230] gitaly[919]: time="2026-11-10T11:57:08.601Z" level=info msg="finished initializing command factory" duration_ms=0 pid=919 gitlab # [ 10.469299] gitaly[919]: time="2026-11-10T11:57:08.601Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-919/git-exec-2420391280.d/git pid=919 gitlab # [ 10.479941] gitaly[919]: time="2026-11-10T11:57:08.612Z" level=info msg="finished detecting git version" duration_ms=11 pid=919 gitlab # [ 10.480554] gitaly[919]: time="2026-11-10T11:57:08.612Z" level=info msg="using Git version" pid=919 version=2.53.g gitlab # [ 10.481443] gitaly[919]: time="2026-11-10T11:57:08.613Z" level=info msg="clearing disk cache object folder" pid=919 storage=default gitlab # [ 10.481905] gitaly[919]: time="2026-11-10T11:57:08.613Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2986301430 pid=919 storage=default gitlab # [ 10.482617] gitaly[919]: time="2026-11-10T11:57:08.613Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2986301430 pid=919 storage=default gitlab # [ 10.483692] gitaly[919]: time="2026-11-10T11:57:08.613Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=919 gitlab # [ 10.484866] gitaly[919]: time="2026-11-10T11:57:08.613Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=919 gitlab # [ 10.485796] gitaly[919]: time="2026-11-10T11:57:08.613Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2986301430 clear_duration_ms=0 pid=919 storage=default gitlab # [ 10.521733] gitaly[919]: time="2026-11-10T11:57:08.653Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=919 gitlab # [ 10.521940] gitaly[919]: time="2026-11-10T11:57:08.653Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=919 schema=unix gitlab # [ 10.522643] gitaly[919]: time="2026-11-10T11:57:08.653Z" level=info msg="listening at address" address=/run/gitaly/gitaly-919/sock.d/intern pid=919 schema=unix gitlab # [ 10.523096] gitaly[919]: time="2026-11-10T11:57:08.653Z" level=info msg="finished tempdir cleaner walk" pid=919 storage=default time_ms=0 gitlab # [ 10.530138] gitaly[919]: time="2026-11-10T11:57:08.653Z" level=info msg="counting repositories" pid=919 gitlab # [ 10.530758] gitaly[919]: time="2026-11-10T11:57:08.662Z" level=info msg="starting to count repositories" pid=919 storage_path=/var/gitlab/state/repositories gitlab # [ 10.533830] gitaly[919]: time="2026-11-10T11:57:08.666Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=919 gitlab # [ 10.534286] gitaly[919]: time="2026-11-10T11:57:08.666Z" level=info msg="Started Gitaly" duration_ms=249 pid=919 gitlab # [ 10.535021] gitaly[919]: time="2026-11-10T11:57:08.666Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=919 gitlab # [ 10.535535] gitaly[919]: time="2026-11-10T11:57:08.666Z" level=info msg="completed counting repositories" counting_duration_ms=3 pid=919 storage_path=/var/gitlab/state/repositories gitlab # [ 10.536476] gitaly[919]: time="2026-11-10T11:57:08.666Z" level=info msg="completed counting all repositories" counting_duration=3 pid=919 gitlab # [ 10.537611] gitaly[919]: time="2026-11-10T11:57:08.666Z" level=info msg="maintenance: daily scheduled" pid=919 scheduled="2026-11-10 12:00:00 +0000 UTC" gitlab # [ 11.404310] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[926]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.411948] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[926]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.435847] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[926]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 13.404639] gitaly[919]: time="2026-11-10T11:57:11.535Z" level=info msg="License database preloaded" duration_ms=3119 pid=919 gitlab # [ 13.627682] dhcpcd[770]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 13.627916] dhcpcd[770]: eth0: adding route to 10.0.2.0/24 gitlab # [ 13.629209] dhcpcd[770]: eth0: adding default route via 10.0.2.2 gitlab # [ 13.728563] systemd[1]: Started DHCP Client. gitlab # [ 13.729829] systemd[1]: Reached target Network is Online. gitlab # [ 43.419411] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[926]: W, [2026-11-10T11:57:41.550752 #926] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 45.329382] postgres[1327]: [1327] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 45.330203] postgres[1327]: [1327] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 45.331226] postgres[1327]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 45.333124] postgres[1327]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 45.335164] postgres[1327]: attidentity AS identity, gitlab # [ 45.336433] postgres[1327]: attgenerated as attgenerated gitlab # [ 45.338338] postgres[1327]: FROM pg_attribute a gitlab # [ 45.340340] postgres[1327]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 45.342308] postgres[1327]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 45.344194] postgres[1327]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 45.346123] postgres[1327]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 45.347630] postgres[1327]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 45.349927] postgres[1327]: ORDER BY a.attnum gitlab # [ 45.352251] postgres[1327]: gitlab # [ 48.418420] gitlab-workhorse[825]: time="2026-11-10T11:57:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 49.118327] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1369]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 49.123297] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1369]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 58.419985] gitlab-workhorse[825]: time="2026-11-10T11:57:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 64.628888] postgres[1444]: [1444] ERROR: relation "table_without_models" does not exist gitlab # [ 64.629356] postgres[1444]: [1444] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 68.420591] gitlab-workhorse[825]: time="2026-11-10T11:58:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 76.794107] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 76.800775] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 76.805737] systemd[1]: Starting gitlab.service... gitlab # [ 77.280302] bundle[1620]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 77.285274] bundle[1620]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 77.290165] bundle[1620]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 78.429440] gitlab-workhorse[825]: time="2026-11-10T11:58:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 80.214342] sidekiq-cluster[1619]: {"severity":"INFO","time":"2026-11-10T11:58:18.326Z","message":"Starting cluster with 1 processes"} gitlab # [ 80.801380] bundle[1620]: {"timestamp":"2026-11-10T11:58:18.903Z","pid":1620,"message":"* Enabling systemd notification integration"} gitlab # [ 80.805321] bundle[1620]: {"timestamp":"2026-11-10T11:58:18.933Z","pid":1620,"message":"Puma starting in cluster mode..."} gitlab # [ 80.808693] bundle[1620]: {"timestamp":"2026-11-10T11:58:18.933Z","pid":1620,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 80.812381] bundle[1620]: {"timestamp":"2026-11-10T11:58:18.933Z","pid":1620,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 80.816725] bundle[1620]: {"timestamp":"2026-11-10T11:58:18.933Z","pid":1620,"message":"* Min threads: 0"} gitlab # [ 80.819743] bundle[1620]: {"timestamp":"2026-11-10T11:58:18.933Z","pid":1620,"message":"* Max threads: 4"} gitlab # [ 80.822758] bundle[1620]: {"timestamp":"2026-11-10T11:58:18.933Z","pid":1620,"message":"* Environment: production"} gitlab # [ 80.826091] bundle[1620]: {"timestamp":"2026-11-10T11:58:18.933Z","pid":1620,"message":"* Master PID: 1620"} gitlab # [ 80.829262] bundle[1620]: {"timestamp":"2026-11-10T11:58:18.933Z","pid":1620,"message":"* Workers: 2"} gitlab # [ 80.832343] bundle[1620]: {"timestamp":"2026-11-10T11:58:18.933Z","pid":1620,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 80.836234] bundle[1620]: {"timestamp":"2026-11-10T11:58:18.933Z","pid":1620,"message":"* Preloading application"} gitlab # [ 88.438901] gitlab-workhorse[825]: time="2026-11-10T11:58:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 98.449060] gitlab-workhorse[825]: time="2026-11-10T11:58:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 106.841331] sidekiq-cluster[1650]: 2026-11-10T11:58:44.972Z pid=1650 tid=wmy INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1650, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>435118080} gitlab # [ 107.033238] sidekiq-cluster[1650]: W, [2026-11-10T11:58:45.165350 #1650] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 107.073989] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:45.205Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 108.271856] bundle[1620]: W, [2026-11-10T11:58:46.403182 #1620] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 108.450175] gitlab-workhorse[825]: time="2026-11-10T11:58:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 108.990906] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.122Z","message":"GitLab reliable fetch activated!"} gitlab # [ 108.992116] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.122Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 109.031186] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.163Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 109.038640] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.170Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 109.046329] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.178Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 109.049394] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.181Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 109.053961] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.186Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 109.058396] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.190Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 109.062560] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.194Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 109.065779] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.197Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 109.069816] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.202Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 109.074145] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.206Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 109.077465] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.209Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 109.082027] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.214Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 109.085368] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.217Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 109.089317] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.221Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 109.093219] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.225Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 109.097269] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.229Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 109.101878] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.234Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 109.107501] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.239Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 109.110084] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.242Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 109.112869] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.244Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 109.116470] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.248Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 109.123752] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.255Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 109.129934] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.261Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 109.133568] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.265Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 109.138519] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.270Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 109.142937] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.275Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 109.147717] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.279Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 109.152636] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.284Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 109.157209] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.289Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 109.159520] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.291Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 109.161529] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.293Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 109.163888] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.296Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 109.166097] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.298Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 109.168395] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.300Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 109.170568] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.302Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 109.172356] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.304Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 109.174537] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.306Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 109.177163] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.309Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 109.179106] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.311Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 109.180927] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.313Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 109.183098] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.315Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 109.184804] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.317Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 109.187060] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.319Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 109.189254] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.321Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 109.191678] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.323Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 109.193638] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.325Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 109.195344] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.327Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 109.197910] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.330Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 109.200262] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.332Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 109.202434] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.334Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 109.204728] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.336Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 109.207144] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.339Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 109.217391] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.349Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 109.219979] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.352Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 109.222494] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.354Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 109.224734] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.356Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 109.226543] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.358Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 109.229131] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.361Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 109.230990] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.363Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 109.233453] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.365Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 109.235234] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.367Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 109.237061] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.369Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 109.239467] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.371Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 109.241956] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.374Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 109.298165] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.430Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 109.301372] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.433Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 109.305080] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.437Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 109.309340] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.441Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 109.314789] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.446Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 109.318313] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.450Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 109.322711] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.454Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 109.326172] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.458Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 109.329775] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.461Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 109.333801] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.466Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 109.336747] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.468Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 109.339983] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.472Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 109.342625] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.474Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 109.345740] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.477Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 109.348392] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.480Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 109.351459] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.483Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 109.353676] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.485Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 109.356504] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.488Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 109.359411] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.491Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 109.362446] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.494Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 109.364868] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.497Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 109.367938] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.500Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 109.370663] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.502Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 109.373510] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.505Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 109.375731] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.507Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 109.378551] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.510Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 109.380734] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.512Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 109.383464] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.515Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 109.385673] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.517Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 109.388039] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.520Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 109.391438] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.523Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 109.397389] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.529Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 109.399854] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.531Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 109.402059] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.533Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 109.409833] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.542Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 109.411487] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.543Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 109.414922] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.547Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 109.420388] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.552Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 109.424540] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.556Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 109.426857] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.559Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 109.432460] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.563Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 109.435406] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.567Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 109.438559] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.570Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 109.443793] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.575Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 109.449078] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:58:47.580Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 118.460137] gitlab-workhorse[825]: time="2026-11-10T11:58:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 128.462818] gitlab-workhorse[825]: time="2026-11-10T11:59:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 133.201096] postgres[2144]: [2144] ERROR: relation "table_without_models" does not exist gitlab # [ 133.203902] postgres[2144]: [2144] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 135.759563] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:13.890Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 135.765699] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:13.897Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 135.770328] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:13.897Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 135.774490] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:13.897Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 135.780663] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:13.912Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 137.265296] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:15.396Z","message":"Cleaning working queues"} gitlab # [ 137.271194] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:15.402Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"0c8c6c7308c49ae8fbb4f678","created_at":"2026-11-10T11:58:14.535Z","trace_propagation_headers":{"sentry-trace":"f97a7f20137a4e5bbccd6233b6dd8fd1-c63b827695314f3f","baggage":"sentry-trace_id=f97a7f20137a4e5bbccd6233b6dd8fd1,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"a41aad2e39288271f1d71741f29a0038","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T11:58:14.542Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-0c8c6c7308c49ae8fbb4f678: start","job_status":"start","queue_duration_s":60.859554,"scheduling_latency_s":60.859627} gitlab # [ 137.464623] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:15.596Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"0c8c6c7308c49ae8fbb4f678","created_at":"2026-11-10T11:58:14.535Z","trace_propagation_headers":{"sentry-trace":"f97a7f20137a4e5bbccd6233b6dd8fd1-c63b827695314f3f","baggage":"sentry-trace_id=f97a7f20137a4e5bbccd6233b6dd8fd1,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"a41aad2e39288271f1d71741f29a0038","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T11:58:14.542Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-0c8c6c7308c49ae8fbb4f678: done: 0.194525 sec","job_status":"done","queue_duration_s":60.859554,"scheduling_latency_s":60.859627,"redis_calls":29,"redis_duration_s":0.007545,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.004554,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001406,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001585,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09185972,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.194525,"completed_at":"2026-11-10T11:59:15.596Z","load_balancing_strategy":"primary","db_duration_s":0.065909,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 138.464075] gitlab-workhorse[825]: time="2026-11-10T11:59:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 138.636796] postgres[2185]: [2185] ERROR: relation "table_without_models" does not exist gitlab # [ 138.637214] postgres[2185]: [2185] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 141.785937] bundle[1620]: {"timestamp":"2026-11-10T11:59:19.917Z","pid":1620,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 141.786952] bundle[1620]: {"timestamp":"2026-11-10T11:59:19.917Z","pid":1620,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 141.789466] bundle[1620]: {"timestamp":"2026-11-10T11:59:19.917Z","pid":1620,"message":"! #\u003cThread:0x000078e2407963c8 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 141.790558] bundle[1620]: {"timestamp":"2026-11-10T11:59:19.917Z","pid":1620,"message":"! #\u003cThread:0x000078e24099d7e8 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 141.792085] bundle[1620]: {"timestamp":"2026-11-10T11:59:19.917Z","pid":1620,"message":"Use Ctrl-C to stop"} gitlab # [ 141.818513] bundle[1620]: {"timestamp":"2026-11-10T11:59:19.918Z","pid":1620,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 142.541545] bundle[1620]: {"timestamp":"2026-11-10T11:59:20.673Z","pid":1620,"message":"- Worker 0 (PID: 2255) booted in 0.69s, phase: 0"} gitlab # [ 142.628386] bundle[1620]: {"timestamp":"2026-11-10T11:59:20.760Z","pid":1620,"message":"- Worker 1 (PID: 2259) booted in 0.75s, phase: 0"} gitlab # [ 142.635600] systemd[1]: Started gitlab.service. gitlab # [ 142.637358] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 142.642455] systemd[1]: Starting setup-gitlab.service... gitlab # [ 142.669131] setup-gitlab-start[2283]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 145.419845] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:11:59:23 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 2712 gitlab # [ 145.424852] setup-gitlab-start[2283]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 145.427465] setup-gitlab-start[2283]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 145.441119] runuser[2305]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 145.839779] setup-gitlab-start[2306]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 145.844852] setup-gitlab-start[2306]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 145.849672] setup-gitlab-start[2306]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 146.451823] setup-gitlab-start[2306]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 146.452292] setup-gitlab-start[2306]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 146.454098] setup-gitlab-start[2306]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 148.491875] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:26.623Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b6f8ffbde84593e182e93aed","created_at":"2026-11-10T11:59:26.606Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53436327ddd95c20a6e5fb2b648e3892","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T11:59:26.622Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b6f8ffbde84593e182e93aed: start","job_status":"start","queue_duration_s":0.00103,"scheduling_latency_s":0.00109} gitlab # [ 148.644731] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:26.776Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"b6f8ffbde84593e182e93aed","created_at":"2026-11-10T11:59:26.606Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53436327ddd95c20a6e5fb2b648e3892","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T11:59:26.622Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-b6f8ffbde84593e182e93aed: done: 0.152751 sec","job_status":"done","queue_duration_s":0.00103,"scheduling_latency_s":0.00109,"redis_calls":10,"redis_duration_s":0.01242,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002579,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008555,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001286,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.007472737,"duration_s":0.152751,"completed_at":"2026-11-10T11:59:26.776Z","load_balancing_strategy":"primary","db_duration_s":0.03594,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.705483] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:26.837Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"0f745916ee36755b402d4554","created_at":"2026-11-10T11:59:26.662Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.690Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-0f745916ee36755b402d4554: start","job_status":"start","queue_duration_s":0.146197,"scheduling_latency_s":0.146225} gitlab # [ 148.948071] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:27.079Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"0f745916ee36755b402d4554","created_at":"2026-11-10T11:59:26.662Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.690Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-0f745916ee36755b402d4554: done: 0.24259 sec","job_status":"done","queue_duration_s":0.146197,"scheduling_latency_s":0.146225,"redis_calls":71,"redis_duration_s":0.07842,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.005941,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001095,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.069082,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002302,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.093883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001043988,"duration_s":0.24259,"completed_at":"2026-11-10T11:59:27.079Z","load_balancing_strategy":"primary","db_duration_s":0.019087,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.991617] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:27.123Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"c080b01505b8c8be23f02163","created_at":"2026-11-10T11:59:26.697Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b5bad72f6e233614af41a58a4a85982","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.701Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-c080b01505b8c8be23f02163: start","job_status":"start","queue_duration_s":0.421272,"scheduling_latency_s":0.421315} gitlab # [ 149.024476] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:27.156Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 149.033405] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:27.165Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"c080b01505b8c8be23f02163","created_at":"2026-11-10T11:59:26.697Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b5bad72f6e233614af41a58a4a85982","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.701Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-c080b01505b8c8be23f02163: done: 0.042044 sec","job_status":"done","queue_duration_s":0.421272,"scheduling_latency_s":0.421315,"redis_calls":10,"redis_duration_s":0.005872,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000535,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003897,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00144,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000271822,"duration_s":0.042044,"completed_at":"2026-11-10T11:59:27.165Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.073863] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:27.205Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"8e039789ca968956db23a4fc","created_at":"2026-11-10T11:59:26.705Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ed2e4aa3d53ddffb0535151efde8482","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.709Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-8e039789ca968956db23a4fc: start","job_status":"start","queue_duration_s":0.496362,"scheduling_latency_s":0.496407} gitlab # [ 149.133078] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:27.264Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"8e039789ca968956db23a4fc","created_at":"2026-11-10T11:59:26.705Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ed2e4aa3d53ddffb0535151efde8482","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.709Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-8e039789ca968956db23a4fc: done: 0.059012 sec","job_status":"done","queue_duration_s":0.496362,"scheduling_latency_s":0.496407,"redis_calls":16,"redis_duration_s":0.005236,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002615,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001906,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000715,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000482463,"duration_s":0.059012,"completed_at":"2026-11-10T11:59:27.264Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016899,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.179870] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:27.311Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"31e7ece2c04cdeb6045c7048","created_at":"2026-11-10T11:59:26.723Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aca786d1bdff5bdfcb1a8da7647e99c2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.733Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-31e7ece2c04cdeb6045c7048: start","job_status":"start","queue_duration_s":0.577326,"scheduling_latency_s":0.577386} gitlab # [ 149.234825] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:27.366Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"31e7ece2c04cdeb6045c7048","created_at":"2026-11-10T11:59:26.723Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aca786d1bdff5bdfcb1a8da7647e99c2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.733Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-31e7ece2c04cdeb6045c7048: done: 0.055507 sec","job_status":"done","queue_duration_s":0.577326,"scheduling_latency_s":0.577386,"redis_calls":19,"redis_duration_s":0.00623,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001366,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000397,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003373,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000432178,"duration_s":0.055507,"completed_at":"2026-11-10T11:59:27.366Z","load_balancing_strategy":"primary","db_duration_s":0.003121,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.236291] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:27.367Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"5ad214f83e3bd411fe446ed1","created_at":"2026-11-10T11:59:26.736Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2ef2cd55f961ba6ab1e7651ffd6c323","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.746Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-5ad214f83e3bd411fe446ed1: start","job_status":"start","queue_duration_s":0.621057,"scheduling_latency_s":0.621076} gitlab # [ 149.367707] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:27.499Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"5ad214f83e3bd411fe446ed1","created_at":"2026-11-10T11:59:26.736Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2ef2cd55f961ba6ab1e7651ffd6c323","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.746Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-5ad214f83e3bd411fe446ed1: done: 0.131251 sec","job_status":"done","queue_duration_s":0.621057,"scheduling_latency_s":0.621076,"redis_calls":14,"redis_duration_s":0.003948,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001334,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001293,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001321,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.063,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.053,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000980572,"exclusive_lock_hold_duration_s":0.047690698999999996,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.131251,"completed_at":"2026-11-10T11:59:27.499Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.063967,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.419645] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:27.551Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"3b2ef770870b497c477982c0","created_at":"2026-11-10T11:59:26.748Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f69cd4d8616b8d343e0df448db28873b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.758Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-3b2ef770870b497c477982c0: start","job_status":"start","queue_duration_s":0.793062,"scheduling_latency_s":0.793122} gitlab # [ 149.468565] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:27.600Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"3b2ef770870b497c477982c0","created_at":"2026-11-10T11:59:26.748Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f69cd4d8616b8d343e0df448db28873b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.758Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-3b2ef770870b497c477982c0: done: 0.04905 sec","job_status":"done","queue_duration_s":0.793062,"scheduling_latency_s":0.793122,"redis_calls":12,"redis_duration_s":0.004098,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000625,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002131,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001342,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000926375,"exclusive_lock_hold_duration_s":0.012718096,"duration_s":0.04905,"completed_at":"2026-11-10T11:59:27.599Z","load_balancing_strategy":"primary","db_duration_s":0.005684,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.508265] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:27.638Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"9efb31c5997b8286ae752c71","created_at":"2026-11-10T11:59:26.764Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"074603acdf8891ef701d016ff869e179","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.778Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-9efb31c5997b8286ae752c71: start","job_status":"start","queue_duration_s":0.860438,"scheduling_latency_s":0.860479} gitlab # [ 149.859337] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:27.991Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"9efb31c5997b8286ae752c71","created_at":"2026-11-10T11:59:26.764Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"074603acdf8891ef701d016ff869e179","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.778Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-9efb31c5997b8286ae752c71: done: 0.352335 sec","job_status":"done","queue_duration_s":0.860438,"scheduling_latency_s":0.860479,"redis_calls":565,"redis_duration_s":0.108752,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000414,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001305,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.107033,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.261647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000301994,"duration_s":0.352335,"completed_at":"2026-11-10T11:59:27.991Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000575,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.921717] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.053Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"014f13bec04d667ad84e7750","created_at":"2026-11-10T11:59:26.785Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ff9e3292793ee188514adc187d33084","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.841Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-014f13bec04d667ad84e7750: start","job_status":"start","queue_duration_s":1.211904,"scheduling_latency_s":1.21193} gitlab # [ 149.972856] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.104Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"014f13bec04d667ad84e7750","created_at":"2026-11-10T11:59:26.785Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ff9e3292793ee188514adc187d33084","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.841Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-014f13bec04d667ad84e7750: done: 0.050861 sec","job_status":"done","queue_duration_s":1.211904,"scheduling_latency_s":1.21193,"redis_calls":10,"redis_duration_s":0.002652,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00045,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001657,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000545,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000465422,"duration_s":0.050861,"completed_at":"2026-11-10T11:59:28.104Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.019236] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.150Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d622fa0ba1ba0c0dbbb89681","created_at":"2026-11-10T11:59:26.869Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d54edff51ec7c1a64989913a9cb134d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.880Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d622fa0ba1ba0c0dbbb89681: start","job_status":"start","queue_duration_s":1.270724,"scheduling_latency_s":1.27078} gitlab # [ 150.066196] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.197Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d622fa0ba1ba0c0dbbb89681","created_at":"2026-11-10T11:59:26.869Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d54edff51ec7c1a64989913a9cb134d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.880Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d622fa0ba1ba0c0dbbb89681: done: 0.047038 sec","job_status":"done","queue_duration_s":1.270724,"scheduling_latency_s":1.27078,"redis_calls":12,"redis_duration_s":0.002879,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001043,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001329,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000507,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000418768,"duration_s":0.047038,"completed_at":"2026-11-10T11:59:28.197Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001459,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.106668] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.238Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"504318a98aad3df6a5eff3ab","created_at":"2026-11-10T11:59:26.896Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6bc35b63344e9a7cf698a9fa2adb114c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.905Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-504318a98aad3df6a5eff3ab: start","job_status":"start","queue_duration_s":1.333211,"scheduling_latency_s":1.333262} gitlab # [ 150.154728] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.285Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"504318a98aad3df6a5eff3ab","created_at":"2026-11-10T11:59:26.896Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6bc35b63344e9a7cf698a9fa2adb114c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.905Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-504318a98aad3df6a5eff3ab: done: 0.047685 sec","job_status":"done","queue_duration_s":1.333211,"scheduling_latency_s":1.333262,"redis_calls":10,"redis_duration_s":0.002937,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0006,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00178,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000557,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000612927,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.047685,"completed_at":"2026-11-10T11:59:28.285Z","load_balancing_strategy":"primary","db_duration_s":0.012622,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.195663] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.327Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1b8f93f4a88468e19e7bba48","created_at":"2026-11-10T11:59:26.920Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"803060d9bed43cbf0b178f865ad056db","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.934Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1b8f93f4a88468e19e7bba48: start","job_status":"start","queue_duration_s":1.392103,"scheduling_latency_s":1.392162} gitlab # [ 150.237148] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1b8f93f4a88468e19e7bba48","created_at":"2026-11-10T11:59:26.920Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"803060d9bed43cbf0b178f865ad056db","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.934Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1b8f93f4a88468e19e7bba48: done: 0.041573 sec","job_status":"done","queue_duration_s":1.392103,"scheduling_latency_s":1.392162,"redis_calls":12,"redis_duration_s":0.004265,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000719,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002007,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001539,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000906261,"exclusive_lock_hold_duration_s":0.005680051,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.041573,"completed_at":"2026-11-10T11:59:28.368Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.278164] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.409Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7c153254872cd6a1bf910fdf","created_at":"2026-11-10T11:59:26.949Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c795ba74cb69b939d36e6b6bb72539d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.957Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7c153254872cd6a1bf910fdf: start","job_status":"start","queue_duration_s":1.451782,"scheduling_latency_s":1.451831} gitlab # [ 150.312233] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.443Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7c153254872cd6a1bf910fdf","created_at":"2026-11-10T11:59:26.949Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c795ba74cb69b939d36e6b6bb72539d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.957Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7c153254872cd6a1bf910fdf: done: 0.034186 sec","job_status":"done","queue_duration_s":1.451782,"scheduling_latency_s":1.451831,"redis_calls":10,"redis_duration_s":0.003537,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000819,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00198,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000738,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000544762,"duration_s":0.034186,"completed_at":"2026-11-10T11:59:28.443Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000847,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.314422] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.446Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"33147ee4b238f14b64402c9c","created_at":"2026-11-10T11:59:26.974Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a6d0b346e014f4d30fb0b27aacc06fc9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.981Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-33147ee4b238f14b64402c9c: start","job_status":"start","queue_duration_s":1.464755,"scheduling_latency_s":1.464783} gitlab # [ 150.358826] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.457Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 150.359670] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.463Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"33147ee4b238f14b64402c9c","created_at":"2026-11-10T11:59:26.974Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a6d0b346e014f4d30fb0b27aacc06fc9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:26.981Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-33147ee4b238f14b64402c9c: done: 0.017047 sec","job_status":"done","queue_duration_s":1.464755,"scheduling_latency_s":1.464783,"redis_calls":10,"redis_duration_s":0.0020080000000000002,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00046,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001206,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000342,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029948,"duration_s":0.017047,"completed_at":"2026-11-10T11:59:28.463Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000536,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.364137] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.464Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"9ce7065d60871ce38104a81f","created_at":"2026-11-10T11:59:26.995Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"febeab317a5e66225cc445bceaed079f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.000Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-9ce7065d60871ce38104a81f: start","job_status":"start","queue_duration_s":1.463956,"scheduling_latency_s":1.463974} gitlab # [ 150.412320] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.474Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 150.413048] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.480Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"9ce7065d60871ce38104a81f","created_at":"2026-11-10T11:59:26.995Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"febeab317a5e66225cc445bceaed079f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.000Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-9ce7065d60871ce38104a81f: done: 0.015696 sec","job_status":"done","queue_duration_s":1.463956,"scheduling_latency_s":1.463974,"redis_calls":10,"redis_duration_s":0.002657,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000458,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001654,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000545,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034306,"duration_s":0.015696,"completed_at":"2026-11-10T11:59:28.480Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00046,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.417742] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.481Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"da84dc1cbe2dc02c9bd30042","created_at":"2026-11-10T11:59:27.008Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8f5f6d2c884179917b4ab154303783c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T11:59:27.014Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-da84dc1cbe2dc02c9bd30042: start","job_status":"start","queue_duration_s":1.466646,"scheduling_latency_s":1.46666} gitlab # [ 150.465748] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.492Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 150.466680] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.506Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"da84dc1cbe2dc02c9bd30042","created_at":"2026-11-10T11:59:27.008Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8f5f6d2c884179917b4ab154303783c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T11:59:27.014Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-da84dc1cbe2dc02c9bd30042: done: 0.024507 sec","job_status":"done","queue_duration_s":1.466646,"scheduling_latency_s":1.46666,"redis_calls":10,"redis_duration_s":0.002913,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000694,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001664,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000555,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000566552,"duration_s":0.024507,"completed_at":"2026-11-10T11:59:28.506Z","load_balancing_strategy":"primary","db_duration_s":0.00047,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.469063] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.508Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"1cd96ee332a2c3e8e9c488f5","created_at":"2026-11-10T11:59:27.022Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4dd739b938df7c8b5a2f4ebf35e42980","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.029Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-1cd96ee332a2c3e8e9c488f5: start","job_status":"start","queue_duration_s":1.478559,"scheduling_latency_s":1.478591} gitlab # [ 150.511708] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.548Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"1cd96ee332a2c3e8e9c488f5","created_at":"2026-11-10T11:59:27.022Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4dd739b938df7c8b5a2f4ebf35e42980","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.029Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-1cd96ee332a2c3e8e9c488f5: done: 0.040729 sec","job_status":"done","queue_duration_s":1.478559,"scheduling_latency_s":1.478591,"redis_calls":13,"redis_duration_s":0.002724,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000432,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001323,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027518,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.012376433,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.040729,"completed_at":"2026-11-10T11:59:28.548Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002772,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 150.555393] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.550Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2bfc431857df1b9358c6f911","created_at":"2026-11-10T11:59:27.034Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31a3777a405f6d4402b0be43ced518b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.041Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2bfc431857df1b9358c6f911: start","job_status":"start","queue_duration_s":1.509171,"scheduling_latency_s":1.509186} gitlab # [ 150.556053] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.560Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 150.556509] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.565Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2bfc431857df1b9358c6f911","created_at":"2026-11-10T11:59:27.034Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31a3777a405f6d4402b0be43ced518b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.041Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2bfc431857df1b9358c6f911: done: 0.014884 sec","job_status":"done","queue_duration_s":1.509171,"scheduling_latency_s":1.509186,"redis_calls":10,"redis_duration_s":0.001335,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000259,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000789,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000287,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000213715,"duration_s":0.014884,"completed_at":"2026-11-10T11:59:28.565Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002034,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.596077] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.566Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"70346ea0be0b9303bc67aa0c","created_at":"2026-11-10T11:59:27.046Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c20c8549a4d46907633cbb0d8441664b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.052Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-70346ea0be0b9303bc67aa0c: start","job_status":"start","queue_duration_s":1.513918,"scheduling_latency_s":1.513933} gitlab # [ 150.596868] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.591Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"70346ea0be0b9303bc67aa0c","created_at":"2026-11-10T11:59:27.046Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c20c8549a4d46907633cbb0d8441664b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.052Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-70346ea0be0b9303bc67aa0c: done: 0.025573 sec","job_status":"done","queue_duration_s":1.513918,"scheduling_latency_s":1.513933,"redis_calls":12,"redis_duration_s":0.002061,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000259,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000815,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000376584,"exclusive_lock_hold_duration_s":0.008479849000000001,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.025573,"completed_at":"2026-11-10T11:59:28.591Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007759,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.625737] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef486eb76d1a9ed5ac9eb733","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef486eb76d1a9ed5ac9eb733: start","job_status":"start","queue_duration_s":1.530336,"scheduling_latency_s":1.530351} gitlab # [ 150.673242] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.614Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef486eb76d1a9ed5ac9eb733","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef486eb76d1a9ed5ac9eb733: done: 0.021328 sec","job_status":"done","queue_duration_s":1.530336,"scheduling_latency_s":1.530351,"redis_calls":16,"redis_duration_s":0.005736,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00368,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000628,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001428,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000218742,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021328,"completed_at":"2026-11-10T11:59:28.614Z","load_balancing_strategy":"primary","db_duration_s":0.002533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.674400] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36363cc0449384eab6250d07","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36363cc0449384eab6250d07: start","job_status":"start","queue_duration_s":1.552602,"scheduling_latency_s":1.552616} gitlab # [ 150.683077] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36363cc0449384eab6250d07","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36363cc0449384eab6250d07: done: 0.010162 sec","job_status":"done","queue_duration_s":1.552602,"scheduling_latency_s":1.552616,"redis_calls":13,"redis_duration_s":0.001573,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006703,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010162,"completed_at":"2026-11-10T11:59:28.625Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.751406] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae95bbccc0aeec3bfbe82955","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae95bbccc0aeec3bfbe82955: start","job_status":"start","queue_duration_s":1.563822,"scheduling_latency_s":1.563837} gitlab # [ 150.752114] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae95bbccc0aeec3bfbe82955","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae95bbccc0aeec3bfbe82955: done: 0.009799 sec","job_status":"done","queue_duration_s":1.563822,"scheduling_latency_s":1.563837,"redis_calls":13,"redis_duration_s":0.001565,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009799,"completed_at":"2026-11-10T11:59:28.636Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.763721] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68456f7d8dd10dbe69f97f69","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68456f7d8dd10dbe69f97f69: start","job_status":"start","queue_duration_s":1.574648,"scheduling_latency_s":1.574662} gitlab # [ 150.774921] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.655Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68456f7d8dd10dbe69f97f69","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68456f7d8dd10dbe69f97f69: done: 0.018384 sec","job_status":"done","queue_duration_s":1.574648,"scheduling_latency_s":1.574662,"redis_calls":13,"redis_duration_s":0.006096000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000729,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005367,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018384,"completed_at":"2026-11-10T11:59:28.655Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.832661] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.657Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11ccf5786951e77066d9b146","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11ccf5786951e77066d9b146: start","job_status":"start","queue_duration_s":1.594479,"scheduling_latency_s":1.594504} gitlab # [ 150.833538] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11ccf5786951e77066d9b146","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11ccf5786951e77066d9b146: done: 0.011962 sec","job_status":"done","queue_duration_s":1.594479,"scheduling_latency_s":1.594504,"redis_calls":13,"redis_duration_s":0.001631,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011962,"completed_at":"2026-11-10T11:59:28.668Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.879672] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.670Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"999c2d52f7dcbd830819114d","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-999c2d52f7dcbd830819114d: start","job_status":"start","queue_duration_s":1.607533,"scheduling_latency_s":1.607548} gitlab # [ 150.880515] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"999c2d52f7dcbd830819114d","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-999c2d52f7dcbd830819114d: done: 0.010744 sec","job_status":"done","queue_duration_s":1.607533,"scheduling_latency_s":1.607548,"redis_calls":13,"redis_duration_s":0.001714,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001173,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007146,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010744,"completed_at":"2026-11-10T11:59:28.680Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.900928] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91259814404821af40670022","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91259814404821af40670022: start","job_status":"start","queue_duration_s":1.61938,"scheduling_latency_s":1.619396} gitlab # [ 150.912253] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91259814404821af40670022","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91259814404821af40670022: done: 0.014136 sec","job_status":"done","queue_duration_s":1.61938,"scheduling_latency_s":1.619396,"redis_calls":13,"redis_duration_s":0.004626,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004082,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014136,"completed_at":"2026-11-10T11:59:28.695Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.954686] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d7e63f65c80c98e0b26e24f","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d7e63f65c80c98e0b26e24f: start","job_status":"start","queue_duration_s":1.634505,"scheduling_latency_s":1.634522} gitlab # [ 150.957520] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d7e63f65c80c98e0b26e24f","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d7e63f65c80c98e0b26e24f: done: 0.010999 sec","job_status":"done","queue_duration_s":1.634505,"scheduling_latency_s":1.634522,"redis_calls":13,"redis_duration_s":0.001592,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010999,"completed_at":"2026-11-10T11:59:28.707Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.016444] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"261a8029ba6d6288ac08e1de","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-261a8029ba6d6288ac08e1de: start","job_status":"start","queue_duration_s":1.64645,"scheduling_latency_s":1.646465} gitlab # [ 151.017439] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.719Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"261a8029ba6d6288ac08e1de","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-261a8029ba6d6288ac08e1de: done: 0.010793 sec","job_status":"done","queue_duration_s":1.64645,"scheduling_latency_s":1.646465,"redis_calls":13,"redis_duration_s":0.001553,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010793,"completed_at":"2026-11-10T11:59:28.719Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.075857] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccf006911b0f2fb925205af6","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccf006911b0f2fb925205af6: start","job_status":"start","queue_duration_s":1.658209,"scheduling_latency_s":1.658227} gitlab # [ 151.076786] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccf006911b0f2fb925205af6","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccf006911b0f2fb925205af6: done: 0.015848 sec","job_status":"done","queue_duration_s":1.658209,"scheduling_latency_s":1.658227,"redis_calls":13,"redis_duration_s":0.0063479999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001031,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005317,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015848,"completed_at":"2026-11-10T11:59:28.736Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.083472] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"309472d332f47b31520d624c","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-309472d332f47b31520d624c: start","job_status":"start","queue_duration_s":1.675111,"scheduling_latency_s":1.675128} gitlab # [ 151.094792] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"309472d332f47b31520d624c","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-309472d332f47b31520d624c: done: 0.01117 sec","job_status":"done","queue_duration_s":1.675111,"scheduling_latency_s":1.675128,"redis_calls":13,"redis_duration_s":0.001873,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000603,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00127,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01117,"completed_at":"2026-11-10T11:59:28.748Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.147224] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f785e9163c14e9db42d5b3cc","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f785e9163c14e9db42d5b3cc: start","job_status":"start","queue_duration_s":1.687256,"scheduling_latency_s":1.687272} gitlab # [ 151.147899] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.761Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f785e9163c14e9db42d5b3cc","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f785e9163c14e9db42d5b3cc: done: 0.011346 sec","job_status":"done","queue_duration_s":1.687256,"scheduling_latency_s":1.687272,"redis_calls":13,"redis_duration_s":0.001658,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007046,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011346,"completed_at":"2026-11-10T11:59:28.761Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.194603] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.762Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b8620d674b97d81dd29032f","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b8620d674b97d81dd29032f: start","job_status":"start","queue_duration_s":1.699615,"scheduling_latency_s":1.69963} gitlab # [ 151.195369] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b8620d674b97d81dd29032f","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b8620d674b97d81dd29032f: done: 0.011267 sec","job_status":"done","queue_duration_s":1.699615,"scheduling_latency_s":1.69963,"redis_calls":13,"redis_duration_s":0.001924,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001167,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011267,"completed_at":"2026-11-10T11:59:28.773Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.238863] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a587f2ac51ba59115a80b5c","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a587f2ac51ba59115a80b5c: start","job_status":"start","queue_duration_s":1.712129,"scheduling_latency_s":1.712147} gitlab # [ 151.239736] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a587f2ac51ba59115a80b5c","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a587f2ac51ba59115a80b5c: done: 0.010936 sec","job_status":"done","queue_duration_s":1.712129,"scheduling_latency_s":1.712147,"redis_calls":13,"redis_duration_s":0.001686,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001121,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010936,"completed_at":"2026-11-10T11:59:28.785Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.265966] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73a1b92cb458f68a01cf0d87","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73a1b92cb458f68a01cf0d87: start","job_status":"start","queue_duration_s":1.724195,"scheduling_latency_s":1.724211} gitlab # [ 151.276939] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73a1b92cb458f68a01cf0d87","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73a1b92cb458f68a01cf0d87: done: 0.011631 sec","job_status":"done","queue_duration_s":1.724195,"scheduling_latency_s":1.724211,"redis_calls":13,"redis_duration_s":0.0019069999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000555,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001352,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011631,"completed_at":"2026-11-10T11:59:28.798Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.340695] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"846be2db59c0785942130a9a","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-846be2db59c0785942130a9a: start","job_status":"start","queue_duration_s":1.73695,"scheduling_latency_s":1.736966} gitlab # [ 151.341549] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.810Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"846be2db59c0785942130a9a","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-846be2db59c0785942130a9a: done: 0.010851 sec","job_status":"done","queue_duration_s":1.73695,"scheduling_latency_s":1.736966,"redis_calls":13,"redis_duration_s":0.001635,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010851,"completed_at":"2026-11-10T11:59:28.810Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.381609] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"451113a04da90f29c9492347","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-451113a04da90f29c9492347: start","job_status":"start","queue_duration_s":1.748924,"scheduling_latency_s":1.748943} gitlab # [ 151.382494] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"451113a04da90f29c9492347","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-451113a04da90f29c9492347: done: 0.011039 sec","job_status":"done","queue_duration_s":1.748924,"scheduling_latency_s":1.748943,"redis_calls":13,"redis_duration_s":0.001581,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011039,"completed_at":"2026-11-10T11:59:28.822Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.435579] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22216c18243a414cdfc7eec7","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22216c18243a414cdfc7eec7: start","job_status":"start","queue_duration_s":1.761095,"scheduling_latency_s":1.761112} gitlab # [ 151.436261] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22216c18243a414cdfc7eec7","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22216c18243a414cdfc7eec7: done: 0.010722 sec","job_status":"done","queue_duration_s":1.761095,"scheduling_latency_s":1.761112,"redis_calls":13,"redis_duration_s":0.0015999999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010722,"completed_at":"2026-11-10T11:59:28.834Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.448104] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55f47e270da52d03758962a5","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55f47e270da52d03758962a5: start","job_status":"start","queue_duration_s":1.772749,"scheduling_latency_s":1.772764} gitlab # [ 151.459500] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55f47e270da52d03758962a5","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55f47e270da52d03758962a5: done: 0.010964 sec","job_status":"done","queue_duration_s":1.772749,"scheduling_latency_s":1.772764,"redis_calls":13,"redis_duration_s":0.001575,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010964,"completed_at":"2026-11-10T11:59:28.846Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.517048] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1131c6ea1e9c6281b694027","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1131c6ea1e9c6281b694027: start","job_status":"start","queue_duration_s":1.78464,"scheduling_latency_s":1.784656} gitlab # [ 151.517834] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.860Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1131c6ea1e9c6281b694027","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1131c6ea1e9c6281b694027: done: 0.013539 sec","job_status":"done","queue_duration_s":1.78464,"scheduling_latency_s":1.784656,"redis_calls":13,"redis_duration_s":0.002088,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000682,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001406,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013539,"completed_at":"2026-11-10T11:59:28.860Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.572336] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"712a92f86b7a68f157fc63b0","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-712a92f86b7a68f157fc63b0: start","job_status":"start","queue_duration_s":1.799137,"scheduling_latency_s":1.799154} gitlab # [ 151.573149] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"712a92f86b7a68f157fc63b0","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-712a92f86b7a68f157fc63b0: done: 0.01142 sec","job_status":"done","queue_duration_s":1.799137,"scheduling_latency_s":1.799154,"redis_calls":13,"redis_duration_s":0.001639,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01142,"completed_at":"2026-11-10T11:59:28.873Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.616768] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c5f8975c0a0f89c869b3cf3","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c5f8975c0a0f89c869b3cf3: start","job_status":"start","queue_duration_s":1.811528,"scheduling_latency_s":1.811545} gitlab # [ 151.617653] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.885Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c5f8975c0a0f89c869b3cf3","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c5f8975c0a0f89c869b3cf3: done: 0.011011 sec","job_status":"done","queue_duration_s":1.811528,"scheduling_latency_s":1.811545,"redis_calls":13,"redis_duration_s":0.001732,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000622,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011011,"completed_at":"2026-11-10T11:59:28.885Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.630548] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.886Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"573960417fd7d3306d7c1659","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-573960417fd7d3306d7c1659: start","job_status":"start","queue_duration_s":1.82353,"scheduling_latency_s":1.823547} gitlab # [ 151.677269] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"573960417fd7d3306d7c1659","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-573960417fd7d3306d7c1659: done: 0.01084 sec","job_status":"done","queue_duration_s":1.82353,"scheduling_latency_s":1.823547,"redis_calls":13,"redis_duration_s":0.0017820000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001242,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01084,"completed_at":"2026-11-10T11:59:28.896Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.678238] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c09e54f8262f9d3a52914cbc","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c09e54f8262f9d3a52914cbc: start","job_status":"start","queue_duration_s":1.835484,"scheduling_latency_s":1.835499} gitlab # [ 151.687593] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c09e54f8262f9d3a52914cbc","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c09e54f8262f9d3a52914cbc: done: 0.010807 sec","job_status":"done","queue_duration_s":1.835484,"scheduling_latency_s":1.835499,"redis_calls":13,"redis_duration_s":0.001569,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010807,"completed_at":"2026-11-10T11:59:28.908Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.737680] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.909Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c99012c9cf86a9fad2dbe583","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c99012c9cf86a9fad2dbe583: start","job_status":"start","queue_duration_s":1.847377,"scheduling_latency_s":1.847393} gitlab # [ 151.738526] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c99012c9cf86a9fad2dbe583","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c99012c9cf86a9fad2dbe583: done: 0.010808 sec","job_status":"done","queue_duration_s":1.847377,"scheduling_latency_s":1.847393,"redis_calls":13,"redis_duration_s":0.001573,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010808,"completed_at":"2026-11-10T11:59:28.920Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.787837] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e847d0062192694443053929","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e847d0062192694443053929: start","job_status":"start","queue_duration_s":1.859296,"scheduling_latency_s":1.859311} gitlab # [ 151.788703] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e847d0062192694443053929","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e847d0062192694443053929: done: 0.010716 sec","job_status":"done","queue_duration_s":1.859296,"scheduling_latency_s":1.859311,"redis_calls":13,"redis_duration_s":0.00162,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010716,"completed_at":"2026-11-10T11:59:28.932Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.829524] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d401b28b747b1ab3b07d8e0","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d401b28b747b1ab3b07d8e0: start","job_status":"start","queue_duration_s":1.871233,"scheduling_latency_s":1.87125} gitlab # [ 151.830227] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.944Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d401b28b747b1ab3b07d8e0","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d401b28b747b1ab3b07d8e0: done: 0.01074 sec","job_status":"done","queue_duration_s":1.871233,"scheduling_latency_s":1.87125,"redis_calls":13,"redis_duration_s":0.0015710000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01074,"completed_at":"2026-11-10T11:59:28.944Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.884329] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ba325ace9b81bcf406a09ba","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ba325ace9b81bcf406a09ba: start","job_status":"start","queue_duration_s":1.88306,"scheduling_latency_s":1.883076} gitlab # [ 151.885068] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ba325ace9b81bcf406a09ba","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ba325ace9b81bcf406a09ba: done: 0.010713 sec","job_status":"done","queue_duration_s":1.88306,"scheduling_latency_s":1.883076,"redis_calls":13,"redis_duration_s":0.0015639999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010713,"completed_at":"2026-11-10T11:59:28.956Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.927636] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c574395bc968232cdb0e680","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c574395bc968232cdb0e680: start","job_status":"start","queue_duration_s":1.895031,"scheduling_latency_s":1.895049} gitlab # [ 151.928266] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c574395bc968232cdb0e680","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c574395bc968232cdb0e680: done: 0.010892 sec","job_status":"done","queue_duration_s":1.895031,"scheduling_latency_s":1.895049,"redis_calls":13,"redis_duration_s":0.001915,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010892,"completed_at":"2026-11-10T11:59:28.968Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.949278] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.969Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"180ebc223e48d8367db7f1ac","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-180ebc223e48d8367db7f1ac: start","job_status":"start","queue_duration_s":1.906928,"scheduling_latency_s":1.906946} gitlab # [ 151.991775] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"180ebc223e48d8367db7f1ac","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-180ebc223e48d8367db7f1ac: done: 0.011533 sec","job_status":"done","queue_duration_s":1.906928,"scheduling_latency_s":1.906946,"redis_calls":13,"redis_duration_s":0.001852,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001266,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011533,"completed_at":"2026-11-10T11:59:28.980Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.994814] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0266b870f69c39c9b6092617","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0266b870f69c39c9b6092617: start","job_status":"start","queue_duration_s":1.919424,"scheduling_latency_s":1.919441} gitlab # [ 152.035816] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0266b870f69c39c9b6092617","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0266b870f69c39c9b6092617: done: 0.010866 sec","job_status":"done","queue_duration_s":1.919424,"scheduling_latency_s":1.919441,"redis_calls":13,"redis_duration_s":0.0017209999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010866,"completed_at":"2026-11-10T11:59:28.992Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.040641] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:28.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"173aef3d21687c265938eaba","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-173aef3d21687c265938eaba: start","job_status":"start","queue_duration_s":1.93124,"scheduling_latency_s":1.931256} gitlab # [ 152.078767] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.004Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"173aef3d21687c265938eaba","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-173aef3d21687c265938eaba: done: 0.010769 sec","job_status":"done","queue_duration_s":1.93124,"scheduling_latency_s":1.931256,"redis_calls":13,"redis_duration_s":0.0016059999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010769,"completed_at":"2026-11-10T11:59:29.004Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.086222] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.005Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2fd59fcc013f22315c15255d","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2fd59fcc013f22315c15255d: start","job_status":"start","queue_duration_s":1.942958,"scheduling_latency_s":1.942974} gitlab # [ 152.097697] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.018Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2fd59fcc013f22315c15255d","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2fd59fcc013f22315c15255d: done: 0.013513 sec","job_status":"done","queue_duration_s":1.942958,"scheduling_latency_s":1.942974,"redis_calls":13,"redis_duration_s":0.001901,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001354,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013513,"completed_at":"2026-11-10T11:59:29.018Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.138799] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56fc93f683e989817535f2bb","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56fc93f683e989817535f2bb: start","job_status":"start","queue_duration_s":1.957567,"scheduling_latency_s":1.957589} gitlab # [ 152.143715] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56fc93f683e989817535f2bb","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56fc93f683e989817535f2bb: done: 0.010991 sec","job_status":"done","queue_duration_s":1.957567,"scheduling_latency_s":1.957589,"redis_calls":13,"redis_duration_s":0.001587,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010991,"completed_at":"2026-11-10T11:59:29.031Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.197899] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"571e83b8839600828cd33351","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-571e83b8839600828cd33351: start","job_status":"start","queue_duration_s":1.969635,"scheduling_latency_s":1.96965} gitlab # [ 152.198766] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"571e83b8839600828cd33351","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-571e83b8839600828cd33351: done: 0.010156 sec","job_status":"done","queue_duration_s":1.969635,"scheduling_latency_s":1.96965,"redis_calls":13,"redis_duration_s":0.0016320000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010156,"completed_at":"2026-11-10T11:59:29.042Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.240299] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.043Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd444c8ca85c590fc069b451","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd444c8ca85c590fc069b451: start","job_status":"start","queue_duration_s":1.98084,"scheduling_latency_s":1.980855} gitlab # [ 152.240691] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.128Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd444c8ca85c590fc069b451","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd444c8ca85c590fc069b451: done: 0.085517 sec","job_status":"done","queue_duration_s":1.98084,"scheduling_latency_s":1.980855,"redis_calls":13,"redis_duration_s":0.002292,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00142,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.085517,"completed_at":"2026-11-10T11:59:29.128Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.284056] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73aadd060d1cb5995ecb96aa","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73aadd060d1cb5995ecb96aa: start","job_status":"start","queue_duration_s":2.067682,"scheduling_latency_s":2.067709} gitlab # [ 152.284734] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73aadd060d1cb5995ecb96aa","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73aadd060d1cb5995ecb96aa: done: 0.012222 sec","job_status":"done","queue_duration_s":2.067682,"scheduling_latency_s":2.067709,"redis_calls":13,"redis_duration_s":0.00168,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001134,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012222,"completed_at":"2026-11-10T11:59:29.142Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.333625] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fce9ab39c2fbe5c5fa564a86","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fce9ab39c2fbe5c5fa564a86: start","job_status":"start","queue_duration_s":2.08087,"scheduling_latency_s":2.080886} gitlab # [ 152.334354] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fce9ab39c2fbe5c5fa564a86","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fce9ab39c2fbe5c5fa564a86: done: 0.01078 sec","job_status":"done","queue_duration_s":2.08087,"scheduling_latency_s":2.080886,"redis_calls":13,"redis_duration_s":0.002169,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001018,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001151,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01078,"completed_at":"2026-11-10T11:59:29.154Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.389982] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe0688754c1adb4f15a0fd93","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe0688754c1adb4f15a0fd93: start","job_status":"start","queue_duration_s":2.092574,"scheduling_latency_s":2.092591} gitlab # [ 152.390779] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe0688754c1adb4f15a0fd93","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe0688754c1adb4f15a0fd93: done: 0.010575 sec","job_status":"done","queue_duration_s":2.092574,"scheduling_latency_s":2.092591,"redis_calls":13,"redis_duration_s":0.001572,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001061,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010575,"completed_at":"2026-11-10T11:59:29.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.437737] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d387702c34d6bd41ce532621","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d387702c34d6bd41ce532621: start","job_status":"start","queue_duration_s":2.104123,"scheduling_latency_s":2.10414} gitlab # [ 152.438759] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.180Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d387702c34d6bd41ce532621","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d387702c34d6bd41ce532621: done: 0.014187 sec","job_status":"done","queue_duration_s":2.104123,"scheduling_latency_s":2.10414,"redis_calls":13,"redis_duration_s":0.002179,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001133,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014187,"completed_at":"2026-11-10T11:59:29.180Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.481920] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8de6c1258a04133f0c5dfa5d","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8de6c1258a04133f0c5dfa5d: start","job_status":"start","queue_duration_s":2.11948,"scheduling_latency_s":2.119498} gitlab # [ 152.482649] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8de6c1258a04133f0c5dfa5d","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8de6c1258a04133f0c5dfa5d: done: 0.010813 sec","job_status":"done","queue_duration_s":2.11948,"scheduling_latency_s":2.119498,"redis_calls":13,"redis_duration_s":0.001564,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001056,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010813,"completed_at":"2026-11-10T11:59:29.192Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.498733] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19e70f519f2a0fcdc87db4c2","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19e70f519f2a0fcdc87db4c2: start","job_status":"start","queue_duration_s":2.131796,"scheduling_latency_s":2.131834} gitlab # [ 152.546354] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19e70f519f2a0fcdc87db4c2","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19e70f519f2a0fcdc87db4c2: done: 0.010536 sec","job_status":"done","queue_duration_s":2.131796,"scheduling_latency_s":2.131834,"redis_calls":13,"redis_duration_s":0.0015890000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010536,"completed_at":"2026-11-10T11:59:29.204Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.547525] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3db88fa2c7d7f3019ce87ced","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3db88fa2c7d7f3019ce87ced: start","job_status":"start","queue_duration_s":2.143272,"scheduling_latency_s":2.143289} gitlab # [ 152.555754] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3db88fa2c7d7f3019ce87ced","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3db88fa2c7d7f3019ce87ced: done: 0.010454 sec","job_status":"done","queue_duration_s":2.143272,"scheduling_latency_s":2.143289,"redis_calls":13,"redis_duration_s":0.001565,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010454,"completed_at":"2026-11-10T11:59:29.216Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.591284] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b80912c502320485b09d3553","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b80912c502320485b09d3553: start","job_status":"start","queue_duration_s":2.154724,"scheduling_latency_s":2.154742} gitlab # [ 152.634149] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b80912c502320485b09d3553","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b80912c502320485b09d3553: done: 0.01498 sec","job_status":"done","queue_duration_s":2.154724,"scheduling_latency_s":2.154742,"redis_calls":13,"redis_duration_s":0.004373,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000622,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003751,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01498,"completed_at":"2026-11-10T11:59:29.232Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.637137] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"186be5ee901dcad74a6b58e7","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-186be5ee901dcad74a6b58e7: start","job_status":"start","queue_duration_s":2.171694,"scheduling_latency_s":2.171712} gitlab # [ 152.683836] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.244Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"186be5ee901dcad74a6b58e7","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-186be5ee901dcad74a6b58e7: done: 0.010804 sec","job_status":"done","queue_duration_s":2.171694,"scheduling_latency_s":2.171712,"redis_calls":13,"redis_duration_s":0.0016510000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010804,"completed_at":"2026-11-10T11:59:29.244Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.685049] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f595ff5fd557d86788ba410","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f595ff5fd557d86788ba410: start","job_status":"start","queue_duration_s":2.183923,"scheduling_latency_s":2.18394} gitlab # [ 152.725060] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.258Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f595ff5fd557d86788ba410","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f595ff5fd557d86788ba410: done: 0.011699 sec","job_status":"done","queue_duration_s":2.183923,"scheduling_latency_s":2.18394,"redis_calls":13,"redis_duration_s":0.001696,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000622,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007236,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011699,"completed_at":"2026-11-10T11:59:29.258Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.728727] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.259Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fd36727e59cbcf1983dfce2","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fd36727e59cbcf1983dfce2: start","job_status":"start","queue_duration_s":2.196574,"scheduling_latency_s":2.196592} gitlab # [ 152.774532] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fd36727e59cbcf1983dfce2","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fd36727e59cbcf1983dfce2: done: 0.010252 sec","job_status":"done","queue_duration_s":2.196574,"scheduling_latency_s":2.196592,"redis_calls":13,"redis_duration_s":0.001579,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006434,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010252,"completed_at":"2026-11-10T11:59:29.269Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.775721] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"264c49873a1aa3f0d3ccbf03","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-264c49873a1aa3f0d3ccbf03: start","job_status":"start","queue_duration_s":2.20776,"scheduling_latency_s":2.207775} gitlab # [ 152.785524] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"264c49873a1aa3f0d3ccbf03","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-264c49873a1aa3f0d3ccbf03: done: 0.01118 sec","job_status":"done","queue_duration_s":2.20776,"scheduling_latency_s":2.207775,"redis_calls":13,"redis_duration_s":0.002069,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000857,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001212,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01118,"completed_at":"2026-11-10T11:59:29.281Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.825826] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e7792a4b9e1323956b9a026","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e7792a4b9e1323956b9a026: start","job_status":"start","queue_duration_s":2.219897,"scheduling_latency_s":2.219913} gitlab # [ 152.830875] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e7792a4b9e1323956b9a026","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e7792a4b9e1323956b9a026: done: 0.014006 sec","job_status":"done","queue_duration_s":2.219897,"scheduling_latency_s":2.219913,"redis_calls":13,"redis_duration_s":0.002521,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000594,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001927,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014006,"completed_at":"2026-11-10T11:59:29.296Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.888711] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c9485d346d5f987858d0d11","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c9485d346d5f987858d0d11: start","job_status":"start","queue_duration_s":2.235148,"scheduling_latency_s":2.235178} gitlab # [ 152.889581] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c9485d346d5f987858d0d11","created_at":"2026-11-10T11:59:26.903Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-cd2e9b7f5e774ac0","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"efa0ca6568303e5009cde2161020fea2","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.062Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c9485d346d5f987858d0d11: done: 0.010397 sec","job_status":"done","queue_duration_s":2.235148,"scheduling_latency_s":2.235178,"redis_calls":13,"redis_duration_s":0.001542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010397,"completed_at":"2026-11-10T11:59:29.308Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.912143] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.308Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e23586a4f1bd4ab3363f5151","created_at":"2026-11-10T11:59:27.064Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba19cd0439993f8014cf00b06e45aa80","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.076Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e23586a4f1bd4ab3363f5151: start","job_status":"start","queue_duration_s":2.231936,"scheduling_latency_s":2.231951} gitlab # [ 152.957129] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.465Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"e23586a4f1bd4ab3363f5151","created_at":"2026-11-10T11:59:27.064Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba19cd0439993f8014cf00b06e45aa80","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.076Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-e23586a4f1bd4ab3363f5151: done: 0.156716 sec","job_status":"done","queue_duration_s":2.231936,"scheduling_latency_s":2.231951,"redis_calls":12,"redis_duration_s":0.001864,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000286,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000811,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000767,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.027,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.101017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000331607,"exclusive_lock_hold_duration_s":0.141409313,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.156716,"completed_at":"2026-11-10T11:59:29.465Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.059375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 152.964271] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"02c56be357ec4c7f6e132b8e","created_at":"2026-11-10T11:59:27.137Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3d2984f1fd5384004befa5ec0e8c0ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T11:59:27.148Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-02c56be357ec4c7f6e132b8e: start","job_status":"start","queue_duration_s":2.318176,"scheduling_latency_s":2.318191} gitlab # [ 152.976560] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.477Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 152.979756] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.481Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"02c56be357ec4c7f6e132b8e","created_at":"2026-11-10T11:59:27.137Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3d2984f1fd5384004befa5ec0e8c0ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T11:59:27.148Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-02c56be357ec4c7f6e132b8e: done: 0.014807 sec","job_status":"done","queue_duration_s":2.318176,"scheduling_latency_s":2.318191,"redis_calls":10,"redis_duration_s":0.00163,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000376,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000951,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000303,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000222096,"duration_s":0.014807,"completed_at":"2026-11-10T11:59:29.481Z","load_balancing_strategy":"primary","db_duration_s":0.002459,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.028271] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.482Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a7d763f3f7f0a4adc20db07b","created_at":"2026-11-10T11:59:27.339Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-847ef45cc9ec4b73","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aca786d1bdff5bdfcb1a8da7647e99c2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.357Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a7d763f3f7f0a4adc20db07b: start","job_status":"start","queue_duration_s":2.124837,"scheduling_latency_s":2.12485} gitlab # [ 153.028941] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.492Z","message":"No existing merge request to be cleaned up."} gitlab # [ 153.029668] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.501Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a7d763f3f7f0a4adc20db07b","created_at":"2026-11-10T11:59:27.339Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-847ef45cc9ec4b73","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aca786d1bdff5bdfcb1a8da7647e99c2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.357Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a7d763f3f7f0a4adc20db07b: done: 0.018749 sec","job_status":"done","queue_duration_s":2.124837,"scheduling_latency_s":2.12485,"redis_calls":13,"redis_duration_s":0.0017590000000000001,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000244,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000675,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212876,"duration_s":0.018749,"completed_at":"2026-11-10T11:59:29.501Z","load_balancing_strategy":"primary","db_duration_s":0.002506,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.102382] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.502Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c98f95c2ea09c675b3211105","created_at":"2026-11-10T11:59:27.339Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-847ef45cc9ec4b73","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aca786d1bdff5bdfcb1a8da7647e99c2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.357Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c98f95c2ea09c675b3211105: start","job_status":"start","queue_duration_s":2.144594,"scheduling_latency_s":2.144609} gitlab # [ 153.103197] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.508Z","message":"No existing merge request to be cleaned up."} gitlab # [ 153.103736] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.513Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c98f95c2ea09c675b3211105","created_at":"2026-11-10T11:59:27.339Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-847ef45cc9ec4b73","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aca786d1bdff5bdfcb1a8da7647e99c2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.357Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c98f95c2ea09c675b3211105: done: 0.011791 sec","job_status":"done","queue_duration_s":2.144594,"scheduling_latency_s":2.144609,"redis_calls":10,"redis_duration_s":0.0013449999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000802,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011791,"completed_at":"2026-11-10T11:59:29.513Z","load_balancing_strategy":"primary","db_duration_s":0.001028,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.111763] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.514Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1dc4b9338ab3f34d53daf738","created_at":"2026-11-10T11:59:27.339Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-847ef45cc9ec4b73","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aca786d1bdff5bdfcb1a8da7647e99c2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.357Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1dc4b9338ab3f34d53daf738: start","job_status":"start","queue_duration_s":2.157319,"scheduling_latency_s":2.157333} gitlab # [ 153.122946] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.523Z","message":"No existing merge request to be cleaned up."} gitlab # [ 153.124584] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.529Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1dc4b9338ab3f34d53daf738","created_at":"2026-11-10T11:59:27.339Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-847ef45cc9ec4b73","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aca786d1bdff5bdfcb1a8da7647e99c2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.357Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1dc4b9338ab3f34d53daf738: done: 0.014286 sec","job_status":"done","queue_duration_s":2.157319,"scheduling_latency_s":2.157333,"redis_calls":10,"redis_duration_s":0.001336,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000798,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007831,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014286,"completed_at":"2026-11-10T11:59:29.529Z","load_balancing_strategy":"primary","db_duration_s":0.004201,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 153.166193] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.530Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ec4f0f92b4abc4460ba35e97","created_at":"2026-11-10T11:59:27.339Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-847ef45cc9ec4b73","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aca786d1bdff5bdfcb1a8da7647e99c2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.357Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ec4f0f92b4abc4460ba35e97: start","job_status":"start","queue_duration_s":2.172494,"scheduling_latency_s":2.172509} gitlab # [ 153.169625] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.536Z","message":"No existing merge request to be cleaned up."} gitlab # [ 153.170966] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T11:59:29.542Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ec4f0f92b4abc4460ba35e97","created_at":"2026-11-10T11:59:27.339Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-847ef45cc9ec4b73","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"aca786d1bdff5bdfcb1a8da7647e99c2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T11:59:27.357Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ec4f0f92b4abc4460ba35e97: done: 0.012027 sec","job_status":"done","queue_duration_s":2.172494,"scheduling_latency_s":2.172509,"redis_calls":10,"redis_duration_s":0.0013289999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000777,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012027,"completed_at":"2026-11-10T11:59:29.542Z","load_balancing_strategy":"primary","db_duration_s":0.001017,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.595414] setup-gitlab-start[2306]: W, [2026-11-10T11:59:50.726800 #2306] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 181.868791] gitaly[919]: time="2026-11-10T12:00:00.000Z" level=info msg="maintenance: daily starting" max_duration=600000000000 pid=919 gitlab # [ 181.869835] gitaly[919]: time="2026-11-10T12:00:00.000Z" level=info msg="maintenance: optimizing repos in storage" pid=919 storage_path=/var/gitlab/state/repositories gitlab # [ 181.871385] gitaly[919]: time="2026-11-10T12:00:00.000Z" level=info msg="maintenance: daily completed" actual_duration="890.456µs" error="" max_duration=600000000000 pid=919 gitlab # [ 181.872882] gitaly[919]: time="2026-11-10T12:00:00.000Z" level=info msg="maintenance: daily scheduled" pid=919 scheduled="2026-11-11 12:00:00 +0000 UTC" gitlab # [ 181.978383] systemd[1]: Starting Logrotate Service... gitlab # [ 182.056778] systemd[1]: logrotate.service: Deactivated successfully. gitlab # [ 182.059946] systemd[1]: Finished Logrotate Service. gitlab # [ 189.103215] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:07.234Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"cae5b60e69b65cd4f605a717","created_at":"2026-11-10T12:00:07.185Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8fc0d9e72e266e41f368b4cdea7f5773","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.230Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-cae5b60e69b65cd4f605a717: start","job_status":"start","queue_duration_s":0.003964,"scheduling_latency_s":0.003991} gitlab # [ 189.392778] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:07.524Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"cae5b60e69b65cd4f605a717","created_at":"2026-11-10T12:00:07.185Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8fc0d9e72e266e41f368b4cdea7f5773","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.230Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-cae5b60e69b65cd4f605a717: done: 0.289388 sec","job_status":"done","queue_duration_s":0.003964,"scheduling_latency_s":0.003991,"redis_calls":12,"redis_duration_s":0.02633,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003373,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.010234,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.012723,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.101642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002695035,"exclusive_lock_hold_duration_s":0.10766955,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.289388,"completed_at":"2026-11-10T12:00:07.524Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.074848,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null} gitlab # [ 189.436905] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:07.568Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"63e35cbacd09e3dbe66fcb19","created_at":"2026-11-10T12:00:07.279Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0b0bab51e3d6cda8a0c758d96caff3d","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.303Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-63e35cbacd09e3dbe66fcb19: start","job_status":"start","queue_duration_s":0.264869,"scheduling_latency_s":0.264914} gitlab # [ 189.487072] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:07.618Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"63e35cbacd09e3dbe66fcb19","created_at":"2026-11-10T12:00:07.279Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0b0bab51e3d6cda8a0c758d96caff3d","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.303Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-63e35cbacd09e3dbe66fcb19: done: 0.050135 sec","job_status":"done","queue_duration_s":0.264869,"scheduling_latency_s":0.264914,"redis_calls":10,"redis_duration_s":0.011041,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007507,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002078,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001948013,"duration_s":0.050135,"completed_at":"2026-11-10T12:00:07.618Z","load_balancing_strategy":"primary","db_duration_s":0.001601,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.524333] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:07.625Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ImportExportProjectCleanupWorker","jid":"9acde072a0cca34bc483ae98","created_at":"2026-11-10T12:00:07.360Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"813ac30523f06aff28535433d4f8cb04","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:c1ab68355154ed2a8f72791841b7337bdce81d68788119e527d7446bb987ef9a","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.376Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ImportExportProjectCleanupWorker JID-9acde072a0cca34bc483ae98: start","job_status":"start","queue_duration_s":0.248682,"scheduling_latency_s":0.248714} gitlab # [ 189.582437] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:07.713Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ImportExportProjectCleanupWorker","jid":"9acde072a0cca34bc483ae98","created_at":"2026-11-10T12:00:07.360Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"813ac30523f06aff28535433d4f8cb04","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:c1ab68355154ed2a8f72791841b7337bdce81d68788119e527d7446bb987ef9a","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.376Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ImportExportProjectCleanupWorker JID-9acde072a0cca34bc483ae98: done: 0.088844 sec","job_status":"done","queue_duration_s":0.248682,"scheduling_latency_s":0.248714,"redis_calls":12,"redis_duration_s":0.011658,"redis_read_bytes":11,"redis_write_bytes":1420,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.008415,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":638,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002456,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000787,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000683886,"duration_s":0.088844,"completed_at":"2026-11-10T12:00:07.713Z","load_balancing_strategy":"primary","db_duration_s":0.037392,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.606384] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:07.719Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"18264c76328a4c1af369701b","created_at":"2026-11-10T12:00:07.389Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9706fa7aa11938a565ffaadf58a4b98","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.399Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-18264c76328a4c1af369701b: start","job_status":"start","queue_duration_s":0.319536,"scheduling_latency_s":0.319572} gitlab # [ 189.676691] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:07.808Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"18264c76328a4c1af369701b","created_at":"2026-11-10T12:00:07.389Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9706fa7aa11938a565ffaadf58a4b98","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.399Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-18264c76328a4c1af369701b: done: 0.089211 sec","job_status":"done","queue_duration_s":0.319536,"scheduling_latency_s":0.319572,"redis_calls":10,"redis_duration_s":0.036865,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000951,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.035256,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000658,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000693384,"duration_s":0.089211,"completed_at":"2026-11-10T12:00:07.808Z","load_balancing_strategy":"primary","db_duration_s":0.008534,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.698095] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:07.812Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::ManageEvidenceWorker","jid":"bc8616d2e10ef839c70af452","created_at":"2026-11-10T12:00:07.406Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33baf4e59c309e19d4688ac75f8ca096","meta.root_caller_id":"Cronjob","meta.feature_category":"release_evidence","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ca33015ef91ab20408351751cf9ad5e1289a7d78257a0117231d08455e951d7f","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.410Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::ManageEvidenceWorker JID-bc8616d2e10ef839c70af452: start","job_status":"start","queue_duration_s":0.401644,"scheduling_latency_s":0.40168} gitlab # [ 189.784932] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:07.916Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::ManageEvidenceWorker","jid":"bc8616d2e10ef839c70af452","created_at":"2026-11-10T12:00:07.406Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33baf4e59c309e19d4688ac75f8ca096","meta.root_caller_id":"Cronjob","meta.feature_category":"release_evidence","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ca33015ef91ab20408351751cf9ad5e1289a7d78257a0117231d08455e951d7f","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.410Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::ManageEvidenceWorker JID-bc8616d2e10ef839c70af452: done: 0.104087 sec","job_status":"done","queue_duration_s":0.401644,"scheduling_latency_s":0.40168,"redis_calls":10,"redis_duration_s":0.041982,"redis_read_bytes":9,"redis_write_bytes":1055,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002426,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.037645,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001911,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":206,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000848432,"duration_s":0.104087,"completed_at":"2026-11-10T12:00:07.916Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016815,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.829997] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:07.923Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f2b6e1ff2fb8867ac52fd341","created_at":"2026-11-10T12:00:07.415Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"402991d5d633b73baae8f39838d50b91","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T12:00:07.416Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f2b6e1ff2fb8867ac52fd341: start","job_status":"start","queue_duration_s":0.506295,"scheduling_latency_s":0.506331} gitlab # [ 189.847243] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:07.978Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f2b6e1ff2fb8867ac52fd341","created_at":"2026-11-10T12:00:07.415Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"402991d5d633b73baae8f39838d50b91","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T12:00:07.416Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f2b6e1ff2fb8867ac52fd341: done: 0.055588 sec","job_status":"done","queue_duration_s":0.506295,"scheduling_latency_s":0.506331,"redis_calls":7,"redis_duration_s":0.016725,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.01462,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002105,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020598,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.055588,"completed_at":"2026-11-10T12:00:07.978Z","load_balancing_strategy":"primary","db_duration_s":0.002719,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.868578] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:07.984Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"7a53b94ad06324794603c664","created_at":"2026-11-10T12:00:07.425Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef702f1d96b765b77344f8067fd185be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.431Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-7a53b94ad06324794603c664: start","job_status":"start","queue_duration_s":0.552978,"scheduling_latency_s":0.55301} gitlab # [ 189.898970] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.030Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"7a53b94ad06324794603c664","created_at":"2026-11-10T12:00:07.425Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef702f1d96b765b77344f8067fd185be","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.431Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-7a53b94ad06324794603c664: done: 0.04625 sec","job_status":"done","queue_duration_s":0.552978,"scheduling_latency_s":0.55301,"redis_calls":12,"redis_duration_s":0.010675,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000848,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005423,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004404,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000450616,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.04625,"completed_at":"2026-11-10T12:00:08.030Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005373,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 189.943761] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.075Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"19c1750f3fff926b92297d47","created_at":"2026-11-10T12:00:07.507Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.511Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-19c1750f3fff926b92297d47: start","job_status":"start","queue_duration_s":0.56373,"scheduling_latency_s":0.563758} gitlab # [ 190.236213] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.367Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"19c1750f3fff926b92297d47","created_at":"2026-11-10T12:00:07.507Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.511Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-19c1750f3fff926b92297d47: done: 0.291684 sec","job_status":"done","queue_duration_s":0.56373,"scheduling_latency_s":0.563758,"redis_calls":60,"redis_duration_s":0.09393800000000001,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.002138,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.088729,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.003071,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.125551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.291684,"completed_at":"2026-11-10T12:00:08.366Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.286865] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.418Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"129c0ddefc16d77c8305c2a8","created_at":"2026-11-10T12:00:07.525Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"acc9f678bbfc5f8b82632b521293f5a9","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.574Z","job_size_bytes":3,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-129c0ddefc16d77c8305c2a8: start","job_status":"start","queue_duration_s":0.844336,"scheduling_latency_s":0.844371} gitlab # [ 190.358284] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.489Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"129c0ddefc16d77c8305c2a8","created_at":"2026-11-10T12:00:07.525Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"acc9f678bbfc5f8b82632b521293f5a9","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.574Z","job_size_bytes":3,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-129c0ddefc16d77c8305c2a8: done: 0.07117 sec","job_status":"done","queue_duration_s":0.844336,"scheduling_latency_s":0.844371,"redis_calls":10,"redis_duration_s":0.020085,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002129,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.016215,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001741,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016386,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002117588,"duration_s":0.07117,"completed_at":"2026-11-10T12:00:08.489Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003989,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.387943] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.512Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"e54453b681d4b67db451065c","created_at":"2026-11-10T12:00:07.587Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"666e5f8056d86b97c9fddce14ab3a68c","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.607Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-e54453b681d4b67db451065c: start","job_status":"start","queue_duration_s":0.904261,"scheduling_latency_s":0.904301} gitlab # [ 190.468865] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.600Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"e54453b681d4b67db451065c","created_at":"2026-11-10T12:00:07.587Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"666e5f8056d86b97c9fddce14ab3a68c","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.607Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-e54453b681d4b67db451065c: done: 0.088238 sec","job_status":"done","queue_duration_s":0.904261,"scheduling_latency_s":0.904301,"redis_calls":12,"redis_duration_s":0.018908,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.004208,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002564,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011134,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002089372,"duration_s":0.088238,"completed_at":"2026-11-10T12:00:08.600Z","load_balancing_strategy":"primary","db_duration_s":0.002122,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.498839] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.614Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BulkImports::StaleImportWorker","jid":"f3647adcc7f176efe2121321","created_at":"2026-11-10T12:00:07.617Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97236e6abc40f0f8a8b0777c70cce28a","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1a00ac23ed20191d6dc4c4907281ed3c2c9a9fd8b3894e6dbd87102304111dab","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.644Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"BulkImports::StaleImportWorker JID-f3647adcc7f176efe2121321: start","job_status":"start","queue_duration_s":0.970593,"scheduling_latency_s":0.970635} gitlab # [ 190.605250] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.735Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BulkImports::StaleImportWorker","jid":"f3647adcc7f176efe2121321","created_at":"2026-11-10T12:00:07.617Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97236e6abc40f0f8a8b0777c70cce28a","meta.root_caller_id":"Cronjob","meta.feature_category":"importers","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1a00ac23ed20191d6dc4c4907281ed3c2c9a9fd8b3894e6dbd87102304111dab","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.644Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"BulkImports::StaleImportWorker JID-f3647adcc7f176efe2121321: done: 0.121334 sec","job_status":"done","queue_duration_s":0.970593,"scheduling_latency_s":0.970635,"redis_calls":10,"redis_duration_s":0.011238999999999999,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001134,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007571,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002534,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032391,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002022044,"duration_s":0.121334,"completed_at":"2026-11-10T12:00:08.735Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020125,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.642938] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.741Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"db71f7d8839ff1d2f652b5a0","created_at":"2026-11-10T12:00:07.648Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ae82152ca40b3ccd5e18dc3014a97c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.653Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-db71f7d8839ff1d2f652b5a0: start","job_status":"start","queue_duration_s":1.0875,"scheduling_latency_s":1.087588} gitlab # [ 190.643916] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.752Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 190.644725] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.759Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"db71f7d8839ff1d2f652b5a0","created_at":"2026-11-10T12:00:07.648Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ae82152ca40b3ccd5e18dc3014a97c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.653Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-db71f7d8839ff1d2f652b5a0: done: 0.018355 sec","job_status":"done","queue_duration_s":1.0875,"scheduling_latency_s":1.087588,"redis_calls":7,"redis_duration_s":0.0027359999999999997,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002136,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0006,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018355,"completed_at":"2026-11-10T12:00:08.759Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.686616] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.761Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"e03c4cd80e80b5348b132a61","created_at":"2026-11-10T12:00:07.659Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa9a13df8e04a64ec26b0d3efd4b6f68","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.662Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-e03c4cd80e80b5348b132a61: start","job_status":"start","queue_duration_s":1.099186,"scheduling_latency_s":1.099212} gitlab # [ 190.735356] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.866Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"e03c4cd80e80b5348b132a61","created_at":"2026-11-10T12:00:07.659Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa9a13df8e04a64ec26b0d3efd4b6f68","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.662Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-e03c4cd80e80b5348b132a61: done: 0.105429 sec","job_status":"done","queue_duration_s":1.099186,"scheduling_latency_s":1.099212,"redis_calls":7,"redis_duration_s":0.0020210000000000002,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0015,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000521,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.087,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.087,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.105429,"completed_at":"2026-11-10T12:00:08.866Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.087535,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.773855] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.905Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"7872a89519d6ecc4b88c4384","created_at":"2026-11-10T12:00:07.680Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"319d47dcfe64f4133f783fda449cdb4e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.694Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-7872a89519d6ecc4b88c4384: start","job_status":"start","queue_duration_s":1.211018,"scheduling_latency_s":1.21112} gitlab # [ 190.826301] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.957Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"7872a89519d6ecc4b88c4384","created_at":"2026-11-10T12:00:07.680Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"319d47dcfe64f4133f783fda449cdb4e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.694Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-7872a89519d6ecc4b88c4384: done: 0.05255 sec","job_status":"done","queue_duration_s":1.211018,"scheduling_latency_s":1.21112,"redis_calls":20,"redis_duration_s":0.005784,"redis_read_bytes":23,"redis_write_bytes":4317,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001862,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.000854,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2022,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.002474,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000594,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000452292,"duration_s":0.05255,"completed_at":"2026-11-10T12:00:08.957Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002261,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.866751] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:08.998Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::PipelineArtifacts::ExpireArtifactsWorker","jid":"df8a8e2471175bc83cb24dc2","created_at":"2026-11-10T12:00:07.708Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4aabed1f9c8f4cb87c9f8f79a278e830","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:35dc78c13bdf78a98c09fc2795030a4961be87ad882288b77a6a3c9be677e4d5","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.718Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::ExpireArtifactsWorker JID-df8a8e2471175bc83cb24dc2: start","job_status":"start","queue_duration_s":1.280042,"scheduling_latency_s":1.280091} gitlab # [ 190.910168] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.041Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::PipelineArtifacts::ExpireArtifactsWorker","jid":"df8a8e2471175bc83cb24dc2","created_at":"2026-11-10T12:00:07.708Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4aabed1f9c8f4cb87c9f8f79a278e830","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:35dc78c13bdf78a98c09fc2795030a4961be87ad882288b77a6a3c9be677e4d5","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.718Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::ExpireArtifactsWorker JID-df8a8e2471175bc83cb24dc2: done: 0.043448 sec","job_status":"done","queue_duration_s":1.280042,"scheduling_latency_s":1.280091,"redis_calls":12,"redis_duration_s":0.004169,"redis_read_bytes":11,"redis_write_bytes":1481,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000677,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002088,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001404,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":548,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000833626,"exclusive_lock_hold_duration_s":0.011723837,"extra.ci_pipeline_artifacts_expire_artifacts_worker.destroyed_pipeline_artifacts_count":0,"duration_s":0.043448,"completed_at":"2026-11-10T12:00:09.041Z","load_balancing_strategy":"primary","db_duration_s":0.00474,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.949726] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.081Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"02e0d331c2cb3f81e31db86d","created_at":"2026-11-10T12:00:07.726Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f38635da70c72dfffe58b60e1b9368a9","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.769Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-02e0d331c2cb3f81e31db86d: start","job_status":"start","queue_duration_s":1.311727,"scheduling_latency_s":1.311767} gitlab # [ 191.002169] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.133Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"02e0d331c2cb3f81e31db86d","created_at":"2026-11-10T12:00:07.726Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f38635da70c72dfffe58b60e1b9368a9","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.769Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-02e0d331c2cb3f81e31db86d: done: 0.052385 sec","job_status":"done","queue_duration_s":1.311727,"scheduling_latency_s":1.311767,"redis_calls":10,"redis_duration_s":0.002428,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001522,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000457,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000423238,"duration_s":0.052385,"completed_at":"2026-11-10T12:00:09.133Z","load_balancing_strategy":"primary","db_duration_s":0.005537,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 191.042448] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.174Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"34f2e3ec428859be8f9d0449","created_at":"2026-11-10T12:00:07.783Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8d6923920d1a1fd2a69b150163ad4fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.788Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-34f2e3ec428859be8f9d0449: start","job_status":"start","queue_duration_s":1.385461,"scheduling_latency_s":1.385507} gitlab # [ 191.079434] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.211Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"34f2e3ec428859be8f9d0449","created_at":"2026-11-10T12:00:07.783Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8d6923920d1a1fd2a69b150163ad4fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.788Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-34f2e3ec428859be8f9d0449: done: 0.037137 sec","job_status":"done","queue_duration_s":1.385461,"scheduling_latency_s":1.385507,"redis_calls":14,"redis_duration_s":0.0045520000000000005,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000578,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002775,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001199,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037137,"completed_at":"2026-11-10T12:00:09.211Z","load_balancing_strategy":"primary","db_duration_s":0.001289,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 191.118284] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.250Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"3ec1d2470ef85e4dc681e4ca","created_at":"2026-11-10T12:00:07.802Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca256aece2446c07233050e6ff0fa549","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.806Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-3ec1d2470ef85e4dc681e4ca: start","job_status":"start","queue_duration_s":1.44402,"scheduling_latency_s":1.444054} gitlab # [ 191.182890] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.313Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"3ec1d2470ef85e4dc681e4ca","created_at":"2026-11-10T12:00:07.802Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca256aece2446c07233050e6ff0fa549","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.806Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-3ec1d2470ef85e4dc681e4ca: done: 0.063544 sec","job_status":"done","queue_duration_s":1.44402,"scheduling_latency_s":1.444054,"redis_calls":9,"redis_duration_s":0.0034929999999999996,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001511,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001982,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001359111,"exclusive_lock_hold_duration_s":0.030096003,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.063544,"completed_at":"2026-11-10T12:00:09.313Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017977,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 191.236240] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.367Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"20c5211116039e7d2bb4f1da","created_at":"2026-11-10T12:00:07.815Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7015d21ef48029633bbfcf512fcba891","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.818Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-20c5211116039e7d2bb4f1da: start","job_status":"start","queue_duration_s":1.549152,"scheduling_latency_s":1.549182} gitlab # [ 191.269311] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.400Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"20c5211116039e7d2bb4f1da","created_at":"2026-11-10T12:00:07.815Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7015d21ef48029633bbfcf512fcba891","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.818Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-20c5211116039e7d2bb4f1da: done: 0.033178 sec","job_status":"done","queue_duration_s":1.549152,"scheduling_latency_s":1.549182,"redis_calls":9,"redis_duration_s":0.0018310000000000002,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000829156,"exclusive_lock_hold_duration_s":0.004693614,"duration_s":0.033178,"completed_at":"2026-11-10T12:00:09.400Z","load_balancing_strategy":"primary","db_duration_s":0.001468,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 191.310599] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.442Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"a4bb5dc9dd31247ff2fc9a24","created_at":"2026-11-10T12:00:07.867Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ddec2060cbf534185acf914b07688da","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.887Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-a4bb5dc9dd31247ff2fc9a24: start","job_status":"start","queue_duration_s":1.554978,"scheduling_latency_s":1.555018} gitlab # [ 191.372687] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.504Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"a4bb5dc9dd31247ff2fc9a24","created_at":"2026-11-10T12:00:07.867Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ddec2060cbf534185acf914b07688da","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.887Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-a4bb5dc9dd31247ff2fc9a24: done: 0.061996 sec","job_status":"done","queue_duration_s":1.554978,"scheduling_latency_s":1.555018,"redis_calls":10,"redis_duration_s":0.0024389999999999998,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000397,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001504,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000538,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000417371,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.002964902},"duration_s":0.061996,"completed_at":"2026-11-10T12:00:09.504Z","load_balancing_strategy":"primary","db_duration_s":0.007526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 191.414697] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.546Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d737da420b6794654f371194","created_at":"2026-11-10T12:00:07.898Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41b6e35603f81bf5fb157e05ef0550d6","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.903Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d737da420b6794654f371194: start","job_status":"start","queue_duration_s":1.642689,"scheduling_latency_s":1.642729} gitlab # [ 191.716866] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.848Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d737da420b6794654f371194","created_at":"2026-11-10T12:00:07.898Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41b6e35603f81bf5fb157e05ef0550d6","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.903Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d737da420b6794654f371194: done: 0.302223 sec","job_status":"done","queue_duration_s":1.642689,"scheduling_latency_s":1.642729,"redis_calls":562,"redis_duration_s":0.09905,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001115,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.097935,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.21926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.302223,"completed_at":"2026-11-10T12:00:09.848Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 191.793651] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.849Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"213b9bc860ae3a27ec6a9e9a","created_at":"2026-11-10T12:00:07.914Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eae86b60c6a354dd17b5cdba403a2326","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.927Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-213b9bc860ae3a27ec6a9e9a: start","job_status":"start","queue_duration_s":1.922664,"scheduling_latency_s":1.922683} gitlab # [ 191.805434] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.880Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"213b9bc860ae3a27ec6a9e9a","created_at":"2026-11-10T12:00:07.914Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eae86b60c6a354dd17b5cdba403a2326","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.927Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-213b9bc860ae3a27ec6a9e9a: done: 0.030528 sec","job_status":"done","queue_duration_s":1.922664,"scheduling_latency_s":1.922683,"redis_calls":11,"redis_duration_s":0.002617,"redis_read_bytes":137,"redis_write_bytes":1243,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000698,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":395,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001467,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000452,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000512076,"duration_s":0.030528,"completed_at":"2026-11-10T12:00:09.880Z","load_balancing_strategy":"primary","db_duration_s":0.000634,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 191.841575] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.882Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7f132f1e05ba04981d4eb815","created_at":"2026-11-10T12:00:07.960Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f63f89020c8072f71395cc56f0e1deb","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.966Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7f132f1e05ba04981d4eb815: start","job_status":"start","queue_duration_s":1.916132,"scheduling_latency_s":1.916163} gitlab # [ 191.854470] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.895Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7f132f1e05ba04981d4eb815","created_at":"2026-11-10T12:00:07.960Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f63f89020c8072f71395cc56f0e1deb","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.966Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7f132f1e05ba04981d4eb815: done: 0.013096 sec","job_status":"done","queue_duration_s":1.916132,"scheduling_latency_s":1.916163,"redis_calls":8,"redis_duration_s":0.0010379999999999999,"redis_read_bytes":133,"redis_write_bytes":730,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000115,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":49,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000681,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013096,"completed_at":"2026-11-10T12:00:09.895Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000689,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 191.855084] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.896Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckCiJobsWorker","jid":"46edbbf0846abac2c5ddefb0","created_at":"2026-11-10T12:00:07.975Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3019f86e6b6e8e607aece195a4239874","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:688518068b75df5544ce76496372eb2176b68f67fc20ec19a768c0a774774cd9","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.986Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckCiJobsWorker JID-46edbbf0846abac2c5ddefb0: start","job_status":"start","queue_duration_s":1.909946,"scheduling_latency_s":1.909962} gitlab # [ 191.864232] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.934Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckCiJobsWorker","jid":"46edbbf0846abac2c5ddefb0","created_at":"2026-11-10T12:00:07.975Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3019f86e6b6e8e607aece195a4239874","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:688518068b75df5544ce76496372eb2176b68f67fc20ec19a768c0a774774cd9","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:07.986Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckCiJobsWorker JID-46edbbf0846abac2c5ddefb0: done: 0.03808 sec","job_status":"done","queue_duration_s":1.909946,"scheduling_latency_s":1.909962,"redis_calls":21,"redis_duration_s":0.002819,"redis_read_bytes":20,"redis_write_bytes":4658,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1380,"redis_queues_calls":3,"redis_queues_duration_s":0.000454,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2315,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":502,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000621,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":461,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.012939354,"duration_s":0.03808,"completed_at":"2026-11-10T12:00:09.934Z","load_balancing_strategy":"primary","db_duration_s":0.007011,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 191.910191] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.935Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PruneOldEventsWorker","jid":"b84dd762f1f6593f2d0a48ed","created_at":"2026-11-10T12:00:08.003Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab9cf0590ef31ff1d0187d5c95c8749e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:aa7847a0e96d8dccea25d7c51fc9fba0c73a3fc691400b9310229c28e3c4fab3","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.014Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"PruneOldEventsWorker JID-b84dd762f1f6593f2d0a48ed: start","job_status":"start","queue_duration_s":1.920679,"scheduling_latency_s":1.920697} gitlab # [ 191.953988] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.955Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PruneOldEventsWorker","jid":"b84dd762f1f6593f2d0a48ed","created_at":"2026-11-10T12:00:08.003Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab9cf0590ef31ff1d0187d5c95c8749e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:aa7847a0e96d8dccea25d7c51fc9fba0c73a3fc691400b9310229c28e3c4fab3","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.014Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"PruneOldEventsWorker JID-b84dd762f1f6593f2d0a48ed: done: 0.019793 sec","job_status":"done","queue_duration_s":1.920679,"scheduling_latency_s":1.920697,"redis_calls":10,"redis_duration_s":0.0013440000000000001,"redis_read_bytes":9,"redis_write_bytes":987,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000263,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000256178,"duration_s":0.019793,"completed_at":"2026-11-10T12:00:09.955Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002433,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 191.955712] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.956Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ac73e7f0ba658000470c923a","created_at":"2026-11-10T12:00:08.026Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bfc300f49cc1aadb43bb77410796a4f7","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.074Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ac73e7f0ba658000470c923a: start","job_status":"start","queue_duration_s":1.881518,"scheduling_latency_s":1.881565} gitlab # [ 191.968942] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.974Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ac73e7f0ba658000470c923a","created_at":"2026-11-10T12:00:08.026Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bfc300f49cc1aadb43bb77410796a4f7","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.074Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ac73e7f0ba658000470c923a: done: 0.018371 sec","job_status":"done","queue_duration_s":1.881518,"scheduling_latency_s":1.881565,"redis_calls":12,"redis_duration_s":0.0018189999999999999,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000806,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000741,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000389714,"exclusive_lock_hold_duration_s":0.003279746,"duration_s":0.018371,"completed_at":"2026-11-10T12:00:09.974Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001435,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.033786] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:09.975Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["[FILTERED]"],"class":"Database::BackgroundOperation::CronEnqueueWorker","jid":"9595dd4bee1755c96abfeb89","created_at":"2026-11-10T12:00:08.083Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"08a54c762665695fb07129eee6957e53","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:38f97efb1ac6a87ded5913833f7d951478e91a6be2a4328abbe154ee188cf7fb","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.097Z","job_size_bytes":101,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CronEnqueueWorker JID-9595dd4bee1755c96abfeb89: start","job_status":"start","queue_duration_s":1.878718,"scheduling_latency_s":1.878734} gitlab # [ 192.125817] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.257Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["[FILTERED]"],"class":"Database::BackgroundOperation::CronEnqueueWorker","jid":"9595dd4bee1755c96abfeb89","created_at":"2026-11-10T12:00:08.083Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"08a54c762665695fb07129eee6957e53","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:38f97efb1ac6a87ded5913833f7d951478e91a6be2a4328abbe154ee188cf7fb","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.097Z","job_size_bytes":101,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CronEnqueueWorker JID-9595dd4bee1755c96abfeb89: done: 0.281456 sec","job_status":"done","queue_duration_s":1.878718,"scheduling_latency_s":1.878734,"redis_calls":10,"redis_duration_s":0.002168,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000366,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001388,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000414,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.216,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.216,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000450057,"duration_s":0.281456,"completed_at":"2026-11-10T12:00:10.257Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.219207,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.171947] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.303Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"d0707c7ad39cba7357121bfc","created_at":"2026-11-10T12:00:08.111Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01528ecb3d537eae82b1514c1b6f8971","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.130Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-d0707c7ad39cba7357121bfc: start","job_status":"start","queue_duration_s":2.172876,"scheduling_latency_s":2.172917} gitlab # [ 192.200600] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.332Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"d0707c7ad39cba7357121bfc","created_at":"2026-11-10T12:00:08.111Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"01528ecb3d537eae82b1514c1b6f8971","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.130Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-d0707c7ad39cba7357121bfc: done: 0.028677 sec","job_status":"done","queue_duration_s":2.172876,"scheduling_latency_s":2.172917,"redis_calls":11,"redis_duration_s":0.003356,"redis_read_bytes":9,"redis_write_bytes":1239,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000691,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001714,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000951,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":298,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000447822,"duration_s":0.028677,"completed_at":"2026-11-10T12:00:10.332Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000581,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.242519] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.373Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"64cd550700e99111514f7c05","created_at":"2026-11-10T12:00:08.141Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"15a052713e7eb6a88dfa77ca10c96e8c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.146Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-64cd550700e99111514f7c05: start","job_status":"start","queue_duration_s":2.227139,"scheduling_latency_s":2.227179} gitlab # [ 192.264150] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.395Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"64cd550700e99111514f7c05","created_at":"2026-11-10T12:00:08.141Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"15a052713e7eb6a88dfa77ca10c96e8c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.146Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-64cd550700e99111514f7c05: done: 0.022266 sec","job_status":"done","queue_duration_s":2.227139,"scheduling_latency_s":2.227179,"redis_calls":8,"redis_duration_s":0.002309,"redis_read_bytes":133,"redis_write_bytes":901,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000314,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001483,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000512,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022266,"completed_at":"2026-11-10T12:00:10.395Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.304106] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.400Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"37dbfc31d8c473d8998e7965","created_at":"2026-11-10T12:00:08.157Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e7c0e7c83c60dd18a0f1dc109f8714e","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.169Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-37dbfc31d8c473d8998e7965: start","job_status":"start","queue_duration_s":2.230968,"scheduling_latency_s":2.231002} gitlab # [ 192.317690] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.435Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"37dbfc31d8c473d8998e7965","created_at":"2026-11-10T12:00:08.157Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e7c0e7c83c60dd18a0f1dc109f8714e","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.169Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-37dbfc31d8c473d8998e7965: done: 0.035147 sec","job_status":"done","queue_duration_s":2.230968,"scheduling_latency_s":2.231002,"redis_calls":10,"redis_duration_s":0.0028799999999999997,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000629,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00172,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000531,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000382451,"duration_s":0.035147,"completed_at":"2026-11-10T12:00:10.435Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004434,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.357094] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.437Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"RepositoryArchiveCacheWorker","jid":"1c54b6ce9fea849ac1500d6d","created_at":"2026-11-10T12:00:08.179Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9ac1c0a31f3e329e0fd12f564820441","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:e95d964ef170969342a5c9d45a52aad8532569d2d839f18acfd130ccc124369d","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.191Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryArchiveCacheWorker JID-1c54b6ce9fea849ac1500d6d: start","job_status":"start","queue_duration_s":2.245643,"scheduling_latency_s":2.245669} gitlab # [ 192.369762] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.452Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"RepositoryArchiveCacheWorker","jid":"1c54b6ce9fea849ac1500d6d","created_at":"2026-11-10T12:00:08.179Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9ac1c0a31f3e329e0fd12f564820441","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:e95d964ef170969342a5c9d45a52aad8532569d2d839f18acfd130ccc124369d","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.191Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryArchiveCacheWorker JID-1c54b6ce9fea849ac1500d6d: done: 0.015191 sec","job_status":"done","queue_duration_s":2.245643,"scheduling_latency_s":2.245669,"redis_calls":12,"redis_duration_s":0.001596,"redis_read_bytes":11,"redis_write_bytes":1392,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000587,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":640,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":206,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002464,"duration_s":0.015191,"completed_at":"2026-11-10T12:00:10.452Z","load_balancing_strategy":"primary","db_duration_s":0.000597,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.407823] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"d67bccb68444bdb1b4fe04e6","created_at":"2026-11-10T12:00:08.203Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce11c2cf9a913aba9c672be5ec50e574","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.215Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-d67bccb68444bdb1b4fe04e6: start","job_status":"start","queue_duration_s":2.238008,"scheduling_latency_s":2.238022} gitlab # [ 192.408203] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.470Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"d67bccb68444bdb1b4fe04e6","created_at":"2026-11-10T12:00:08.203Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce11c2cf9a913aba9c672be5ec50e574","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.215Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-d67bccb68444bdb1b4fe04e6: done: 0.017381 sec","job_status":"done","queue_duration_s":2.238008,"scheduling_latency_s":2.238022,"redis_calls":10,"redis_duration_s":0.0013289999999999999,"redis_read_bytes":9,"redis_write_bytes":1097,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000257,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":309,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000819,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000253,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":218,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01029,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000223492,"duration_s":0.017381,"completed_at":"2026-11-10T12:00:10.470Z","load_balancing_strategy":"primary","db_duration_s":0.002233,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.448737] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.471Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"575e448534675b70a4c4fdb4","created_at":"2026-11-10T12:00:08.241Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.251Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-575e448534675b70a4c4fdb4: start","job_status":"start","queue_duration_s":2.220047,"scheduling_latency_s":2.22006} gitlab # [ 192.449641] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.500Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"575e448534675b70a4c4fdb4","created_at":"2026-11-10T12:00:08.241Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.251Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-575e448534675b70a4c4fdb4: done: 0.029272 sec","job_status":"done","queue_duration_s":2.220047,"scheduling_latency_s":2.22006,"redis_calls":28,"redis_duration_s":0.003611,"redis_read_bytes":36,"redis_write_bytes":10603,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000512,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.000237,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8315,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.002019,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000843,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00020673,"duration_s":0.029272,"completed_at":"2026-11-10T12:00:10.500Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002018,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.489294] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.501Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"2cd54b37b57f8f3b5aa7b0d3","created_at":"2026-11-10T12:00:08.261Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"235447803f5fbd529c425176392a2a6a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.264Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-2cd54b37b57f8f3b5aa7b0d3: start","job_status":"start","queue_duration_s":2.23695,"scheduling_latency_s":2.236964} gitlab # [ 192.489928] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.515Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"2cd54b37b57f8f3b5aa7b0d3","created_at":"2026-11-10T12:00:08.261Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"235447803f5fbd529c425176392a2a6a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.264Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-2cd54b37b57f8f3b5aa7b0d3: done: 0.013871 sec","job_status":"done","queue_duration_s":2.23695,"scheduling_latency_s":2.236964,"redis_calls":7,"redis_duration_s":0.000956,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000706,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00025,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006515,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.013871,"completed_at":"2026-11-10T12:00:10.515Z","load_balancing_strategy":"primary","db_duration_s":0.003899,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.529115] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.516Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d20122b83668f021ce9a7b2e","created_at":"2026-11-10T12:00:08.277Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52153b640a21597ff50b5f4c710c247b","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.281Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d20122b83668f021ce9a7b2e: start","job_status":"start","queue_duration_s":2.235131,"scheduling_latency_s":2.235154} gitlab # [ 192.530593] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d20122b83668f021ce9a7b2e","created_at":"2026-11-10T12:00:08.277Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52153b640a21597ff50b5f4c710c247b","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.281Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d20122b83668f021ce9a7b2e: done: 0.01063 sec","job_status":"done","queue_duration_s":2.235131,"scheduling_latency_s":2.235154,"redis_calls":9,"redis_duration_s":0.0012430000000000002,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000581,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000329371,"exclusive_lock_hold_duration_s":0.00187873,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.01063,"completed_at":"2026-11-10T12:00:10.527Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.577108] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.528Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"cc5d27a951bbfeed83b69d5e","created_at":"2026-11-10T12:00:08.318Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.327Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-cc5d27a951bbfeed83b69d5e: start","job_status":"start","queue_duration_s":2.200576,"scheduling_latency_s":2.200589} gitlab # [ 192.578940] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.588Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"cc5d27a951bbfeed83b69d5e","created_at":"2026-11-10T12:00:08.318Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.327Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-cc5d27a951bbfeed83b69d5e: done: 0.060216 sec","job_status":"done","queue_duration_s":2.200576,"scheduling_latency_s":2.200589,"redis_calls":65,"redis_duration_s":0.008446,"redis_read_bytes":115,"redis_write_bytes":41332,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000504,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_calls":2,"redis_queues_duration_s":0.000417,"redis_queues_read_bytes":4,"redis_queues_write_bytes":37035,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.007149,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3374,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000376,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000241372,"duration_s":0.060216,"completed_at":"2026-11-10T12:00:10.588Z","load_balancing_strategy":"primary","db_duration_s":0.000568,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.646810] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"30c79009637b6dff06936724","created_at":"2026-11-10T12:00:08.337Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7b54c7f403f2c716150dd0b1c752cff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.345Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-30c79009637b6dff06936724: start","job_status":"start","queue_duration_s":2.244173,"scheduling_latency_s":2.244188} gitlab # [ 192.647688] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.598Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"30c79009637b6dff06936724","created_at":"2026-11-10T12:00:08.337Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7b54c7f403f2c716150dd0b1c752cff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.345Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-30c79009637b6dff06936724: done: 0.008772 sec","job_status":"done","queue_duration_s":2.244173,"scheduling_latency_s":2.244188,"redis_calls":7,"redis_duration_s":0.000946,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000697,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008772,"completed_at":"2026-11-10T12:00:10.598Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.688771] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b1263ccd8e2fbca7860d177","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b1263ccd8e2fbca7860d177: start","job_status":"start","queue_duration_s":2.250683,"scheduling_latency_s":2.250697} gitlab # [ 192.689724] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b1263ccd8e2fbca7860d177","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b1263ccd8e2fbca7860d177: done: 0.010185 sec","job_status":"done","queue_duration_s":2.250683,"scheduling_latency_s":2.250697,"redis_calls":13,"redis_duration_s":0.001778,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001278,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010185,"completed_at":"2026-11-10T12:00:10.609Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.733957] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.610Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e1c00a3f8e3c0e5d73ec29c","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e1c00a3f8e3c0e5d73ec29c: start","job_status":"start","queue_duration_s":2.261729,"scheduling_latency_s":2.261744} gitlab # [ 192.734725] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.620Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e1c00a3f8e3c0e5d73ec29c","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e1c00a3f8e3c0e5d73ec29c: done: 0.009856 sec","job_status":"done","queue_duration_s":2.261729,"scheduling_latency_s":2.261744,"redis_calls":13,"redis_duration_s":0.0015990000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009856,"completed_at":"2026-11-10T12:00:10.620Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.756598] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3017c09375ee320b768eaae4","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3017c09375ee320b768eaae4: start","job_status":"start","queue_duration_s":2.272457,"scheduling_latency_s":2.272471} gitlab # [ 192.795730] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3017c09375ee320b768eaae4","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3017c09375ee320b768eaae4: done: 0.009859 sec","job_status":"done","queue_duration_s":2.272457,"scheduling_latency_s":2.272471,"redis_calls":13,"redis_duration_s":0.0015899999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009859,"completed_at":"2026-11-10T12:00:10.630Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.801903] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"741368441307bc41d5bdbc2e","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-741368441307bc41d5bdbc2e: start","job_status":"start","queue_duration_s":2.291122,"scheduling_latency_s":2.291141} gitlab # [ 192.841708] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"741368441307bc41d5bdbc2e","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-741368441307bc41d5bdbc2e: done: 0.010123 sec","job_status":"done","queue_duration_s":2.291122,"scheduling_latency_s":2.291141,"redis_calls":13,"redis_duration_s":0.0015769999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010123,"completed_at":"2026-11-10T12:00:10.649Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.882722] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f66a4488338bd0847ce2d19","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f66a4488338bd0847ce2d19: start","job_status":"start","queue_duration_s":2.302118,"scheduling_latency_s":2.302133} gitlab # [ 192.883478] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f66a4488338bd0847ce2d19","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f66a4488338bd0847ce2d19: done: 0.009834 sec","job_status":"done","queue_duration_s":2.302118,"scheduling_latency_s":2.302133,"redis_calls":13,"redis_duration_s":0.0016259999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001122,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009834,"completed_at":"2026-11-10T12:00:10.660Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.929711] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.661Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e800fe3aa19e02c59ec014c7","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e800fe3aa19e02c59ec014c7: start","job_status":"start","queue_duration_s":2.312834,"scheduling_latency_s":2.312848} gitlab # [ 192.930417] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e800fe3aa19e02c59ec014c7","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e800fe3aa19e02c59ec014c7: done: 0.009724 sec","job_status":"done","queue_duration_s":2.312834,"scheduling_latency_s":2.312848,"redis_calls":13,"redis_duration_s":0.0016220000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009724,"completed_at":"2026-11-10T12:00:10.671Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 192.968470] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.672Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52736daea1f0fb4e73e30c67","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52736daea1f0fb4e73e30c67: start","job_status":"start","queue_duration_s":2.323473,"scheduling_latency_s":2.323487} gitlab # [ 192.969473] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52736daea1f0fb4e73e30c67","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52736daea1f0fb4e73e30c67: done: 0.009619 sec","job_status":"done","queue_duration_s":2.323473,"scheduling_latency_s":2.323487,"redis_calls":13,"redis_duration_s":0.0015439999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009619,"completed_at":"2026-11-10T12:00:10.681Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.009130] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.682Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fac2d2e81a42ca630fba6a6","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fac2d2e81a42ca630fba6a6: start","job_status":"start","queue_duration_s":2.333982,"scheduling_latency_s":2.333996} gitlab # [ 193.009847] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fac2d2e81a42ca630fba6a6","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fac2d2e81a42ca630fba6a6: done: 0.009609 sec","job_status":"done","queue_duration_s":2.333982,"scheduling_latency_s":2.333996,"redis_calls":13,"redis_duration_s":0.0015899999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009609,"completed_at":"2026-11-10T12:00:10.692Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.051131] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1517165b366f30b630093272","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1517165b366f30b630093272: start","job_status":"start","queue_duration_s":2.344466,"scheduling_latency_s":2.34448} gitlab # [ 193.051888] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1517165b366f30b630093272","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1517165b366f30b630093272: done: 0.009642 sec","job_status":"done","queue_duration_s":2.344466,"scheduling_latency_s":2.34448,"redis_calls":13,"redis_duration_s":0.001582,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009642,"completed_at":"2026-11-10T12:00:10.702Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.090316] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.703Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c8dd3906e6677be6a063b99","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c8dd3906e6677be6a063b99: start","job_status":"start","queue_duration_s":2.354965,"scheduling_latency_s":2.354978} gitlab # [ 193.091037] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c8dd3906e6677be6a063b99","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c8dd3906e6677be6a063b99: done: 0.009855 sec","job_status":"done","queue_duration_s":2.354965,"scheduling_latency_s":2.354978,"redis_calls":13,"redis_duration_s":0.001558,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009855,"completed_at":"2026-11-10T12:00:10.713Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.118773] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0884480d485df470fb17ae38","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0884480d485df470fb17ae38: start","job_status":"start","queue_duration_s":2.365676,"scheduling_latency_s":2.36569} gitlab # [ 193.165725] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0884480d485df470fb17ae38","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0884480d485df470fb17ae38: done: 0.009793 sec","job_status":"done","queue_duration_s":2.365676,"scheduling_latency_s":2.36569,"redis_calls":13,"redis_duration_s":0.001549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009793,"completed_at":"2026-11-10T12:00:10.724Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.166839] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a2ff983b52fd4dbd9c47607","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a2ff983b52fd4dbd9c47607: start","job_status":"start","queue_duration_s":2.376354,"scheduling_latency_s":2.376367} gitlab # [ 193.173699] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.734Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a2ff983b52fd4dbd9c47607","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a2ff983b52fd4dbd9c47607: done: 0.009495 sec","job_status":"done","queue_duration_s":2.376354,"scheduling_latency_s":2.376367,"redis_calls":13,"redis_duration_s":0.0015609999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009495,"completed_at":"2026-11-10T12:00:10.734Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.219752] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20962d2ead728b66b9e25263","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20962d2ead728b66b9e25263: start","job_status":"start","queue_duration_s":2.386686,"scheduling_latency_s":2.386727} gitlab # [ 193.220577] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20962d2ead728b66b9e25263","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20962d2ead728b66b9e25263: done: 0.00944 sec","job_status":"done","queue_duration_s":2.386686,"scheduling_latency_s":2.386727,"redis_calls":13,"redis_duration_s":0.0015769999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00944,"completed_at":"2026-11-10T12:00:10.744Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.267215] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25947a598a3bac7412e50fb3","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25947a598a3bac7412e50fb3: start","job_status":"start","queue_duration_s":2.396978,"scheduling_latency_s":2.396993} gitlab # [ 193.267971] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25947a598a3bac7412e50fb3","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25947a598a3bac7412e50fb3: done: 0.009329 sec","job_status":"done","queue_duration_s":2.396978,"scheduling_latency_s":2.396993,"redis_calls":13,"redis_duration_s":0.001559,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009329,"completed_at":"2026-11-10T12:00:10.754Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.308777] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77c647f512be0356dd399d3d","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77c647f512be0356dd399d3d: start","job_status":"start","queue_duration_s":2.407176,"scheduling_latency_s":2.40719} gitlab # [ 193.309468] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77c647f512be0356dd399d3d","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77c647f512be0356dd399d3d: done: 0.009218 sec","job_status":"done","queue_duration_s":2.407176,"scheduling_latency_s":2.40719,"redis_calls":13,"redis_duration_s":0.001552,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009218,"completed_at":"2026-11-10T12:00:10.764Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.359995] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa8c3a326bc45f4017ddb838","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa8c3a326bc45f4017ddb838: start","job_status":"start","queue_duration_s":2.417249,"scheduling_latency_s":2.417262} gitlab # [ 193.360692] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aa8c3a326bc45f4017ddb838","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aa8c3a326bc45f4017ddb838: done: 0.009554 sec","job_status":"done","queue_duration_s":2.417249,"scheduling_latency_s":2.417262,"redis_calls":13,"redis_duration_s":0.001738,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001228,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005896,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009554,"completed_at":"2026-11-10T12:00:10.775Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.401569] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d11701f53e31cd7cd3a87d2","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d11701f53e31cd7cd3a87d2: start","job_status":"start","queue_duration_s":2.427632,"scheduling_latency_s":2.427646} gitlab # [ 193.402192] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d11701f53e31cd7cd3a87d2","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d11701f53e31cd7cd3a87d2: done: 0.009198 sec","job_status":"done","queue_duration_s":2.427632,"scheduling_latency_s":2.427646,"redis_calls":13,"redis_duration_s":0.0014739999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009198,"completed_at":"2026-11-10T12:00:10.785Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.429152] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c4682dd93843a23882a6a75","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c4682dd93843a23882a6a75: start","job_status":"start","queue_duration_s":2.43767,"scheduling_latency_s":2.437684} gitlab # [ 193.473952] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c4682dd93843a23882a6a75","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c4682dd93843a23882a6a75: done: 0.009249 sec","job_status":"done","queue_duration_s":2.43767,"scheduling_latency_s":2.437684,"redis_calls":13,"redis_duration_s":0.001543,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009249,"completed_at":"2026-11-10T12:00:10.795Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.474904] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"121f6fdccc9cbfbce3a82d30","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-121f6fdccc9cbfbce3a82d30: start","job_status":"start","queue_duration_s":2.447759,"scheduling_latency_s":2.447773} gitlab # [ 193.484628] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"121f6fdccc9cbfbce3a82d30","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-121f6fdccc9cbfbce3a82d30: done: 0.009497 sec","job_status":"done","queue_duration_s":2.447759,"scheduling_latency_s":2.447773,"redis_calls":13,"redis_duration_s":0.0015249999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009497,"completed_at":"2026-11-10T12:00:10.805Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.549707] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b24cf3c374f814864bd9e10d","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b24cf3c374f814864bd9e10d: start","job_status":"start","queue_duration_s":2.458122,"scheduling_latency_s":2.458137} gitlab # [ 193.550520] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b24cf3c374f814864bd9e10d","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b24cf3c374f814864bd9e10d: done: 0.009309 sec","job_status":"done","queue_duration_s":2.458122,"scheduling_latency_s":2.458137,"redis_calls":13,"redis_duration_s":0.0015869999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009309,"completed_at":"2026-11-10T12:00:10.816Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.595772] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8527a37122a0d68fc63f2a4","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8527a37122a0d68fc63f2a4: start","job_status":"start","queue_duration_s":2.468301,"scheduling_latency_s":2.468315} gitlab # [ 193.596732] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8527a37122a0d68fc63f2a4","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8527a37122a0d68fc63f2a4: done: 0.009305 sec","job_status":"done","queue_duration_s":2.468301,"scheduling_latency_s":2.468315,"redis_calls":13,"redis_duration_s":0.001543,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009305,"completed_at":"2026-11-10T12:00:10.826Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.637680] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f4325d505038216160a2517","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f4325d505038216160a2517: start","job_status":"start","queue_duration_s":2.478461,"scheduling_latency_s":2.478476} gitlab # [ 193.638528] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.836Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f4325d505038216160a2517","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f4325d505038216160a2517: done: 0.009231 sec","job_status":"done","queue_duration_s":2.478461,"scheduling_latency_s":2.478476,"redis_calls":13,"redis_duration_s":0.00151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009231,"completed_at":"2026-11-10T12:00:10.836Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.652677] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40491bbafaf4c5f666f3afe8","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40491bbafaf4c5f666f3afe8: start","job_status":"start","queue_duration_s":2.48854,"scheduling_latency_s":2.488554} gitlab # [ 193.692800] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40491bbafaf4c5f666f3afe8","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40491bbafaf4c5f666f3afe8: done: 0.009344 sec","job_status":"done","queue_duration_s":2.48854,"scheduling_latency_s":2.488554,"redis_calls":13,"redis_duration_s":0.00148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009344,"completed_at":"2026-11-10T12:00:10.846Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.696826] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ededf07d3d0576448771ee72","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ededf07d3d0576448771ee72: start","job_status":"start","queue_duration_s":2.498804,"scheduling_latency_s":2.498819} gitlab # [ 193.735032] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ededf07d3d0576448771ee72","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ededf07d3d0576448771ee72: done: 0.009405 sec","job_status":"done","queue_duration_s":2.498804,"scheduling_latency_s":2.498819,"redis_calls":13,"redis_duration_s":0.0015439999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001027,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009405,"completed_at":"2026-11-10T12:00:10.856Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.775732] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d120b8dc2fad76fe83b58b7b","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d120b8dc2fad76fe83b58b7b: start","job_status":"start","queue_duration_s":2.509008,"scheduling_latency_s":2.509021} gitlab # [ 193.776566] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d120b8dc2fad76fe83b58b7b","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d120b8dc2fad76fe83b58b7b: done: 0.013544 sec","job_status":"done","queue_duration_s":2.509008,"scheduling_latency_s":2.509021,"redis_calls":13,"redis_duration_s":0.001599,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013544,"completed_at":"2026-11-10T12:00:10.871Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.817740] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af47cedbaa14415c95d3ef13","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af47cedbaa14415c95d3ef13: start","job_status":"start","queue_duration_s":2.523437,"scheduling_latency_s":2.523452} gitlab # [ 193.818452] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af47cedbaa14415c95d3ef13","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af47cedbaa14415c95d3ef13: done: 0.011139 sec","job_status":"done","queue_duration_s":2.523437,"scheduling_latency_s":2.523452,"redis_calls":13,"redis_duration_s":0.001587,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011139,"completed_at":"2026-11-10T12:00:10.883Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.829835] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e5e4f326ab0449904a5617f","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e5e4f326ab0449904a5617f: start","job_status":"start","queue_duration_s":2.535451,"scheduling_latency_s":2.535465} gitlab # [ 193.873352] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e5e4f326ab0449904a5617f","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e5e4f326ab0449904a5617f: done: 0.009197 sec","job_status":"done","queue_duration_s":2.535451,"scheduling_latency_s":2.535465,"redis_calls":13,"redis_duration_s":0.001492,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001005,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009197,"completed_at":"2026-11-10T12:00:10.893Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.874580] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddca6c018d9147c06358de1e","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddca6c018d9147c06358de1e: start","job_status":"start","queue_duration_s":2.545528,"scheduling_latency_s":2.545542} gitlab # [ 193.886191] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddca6c018d9147c06358de1e","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddca6c018d9147c06358de1e: done: 0.009345 sec","job_status":"done","queue_duration_s":2.545528,"scheduling_latency_s":2.545542,"redis_calls":13,"redis_duration_s":0.00154,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009345,"completed_at":"2026-11-10T12:00:10.903Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 193.949074] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f45e3a315f5d5b5742e2972","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f45e3a315f5d5b5742e2972: start","job_status":"start","queue_duration_s":2.55573,"scheduling_latency_s":2.555744} gitlab # [ 193.949817] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f45e3a315f5d5b5742e2972","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f45e3a315f5d5b5742e2972: done: 0.009692 sec","job_status":"done","queue_duration_s":2.55573,"scheduling_latency_s":2.555744,"redis_calls":13,"redis_duration_s":0.001553,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009692,"completed_at":"2026-11-10T12:00:10.913Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.000595] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fdd6ea558af9a766095aa4a","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fdd6ea558af9a766095aa4a: start","job_status":"start","queue_duration_s":2.566273,"scheduling_latency_s":2.566286} gitlab # [ 194.001456] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fdd6ea558af9a766095aa4a","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fdd6ea558af9a766095aa4a: done: 0.009261 sec","job_status":"done","queue_duration_s":2.566273,"scheduling_latency_s":2.566286,"redis_calls":13,"redis_duration_s":0.001554,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009261,"completed_at":"2026-11-10T12:00:10.924Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.041754] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bacc39fa092594848e35db8","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bacc39fa092594848e35db8: start","job_status":"start","queue_duration_s":2.5764,"scheduling_latency_s":2.576414} gitlab # [ 194.042423] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bacc39fa092594848e35db8","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bacc39fa092594848e35db8: done: 0.009274 sec","job_status":"done","queue_duration_s":2.5764,"scheduling_latency_s":2.576414,"redis_calls":13,"redis_duration_s":0.001535,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009274,"completed_at":"2026-11-10T12:00:10.934Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.084943] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.935Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d192ac515b0b35b10709bf81","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d192ac515b0b35b10709bf81: start","job_status":"start","queue_duration_s":2.586544,"scheduling_latency_s":2.586558} gitlab # [ 194.085678] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.944Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d192ac515b0b35b10709bf81","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d192ac515b0b35b10709bf81: done: 0.009365 sec","job_status":"done","queue_duration_s":2.586544,"scheduling_latency_s":2.586558,"redis_calls":13,"redis_duration_s":0.001581,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001081,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009365,"completed_at":"2026-11-10T12:00:10.944Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.103819] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44143cb14e0a8d12401bec48","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44143cb14e0a8d12401bec48: start","job_status":"start","queue_duration_s":2.59679,"scheduling_latency_s":2.596803} gitlab # [ 194.144611] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44143cb14e0a8d12401bec48","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44143cb14e0a8d12401bec48: done: 0.009332 sec","job_status":"done","queue_duration_s":2.59679,"scheduling_latency_s":2.596803,"redis_calls":13,"redis_duration_s":0.0015760000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009332,"completed_at":"2026-11-10T12:00:10.954Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.149740] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28ff3caf5a2d806b263bb0e6","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28ff3caf5a2d806b263bb0e6: start","job_status":"start","queue_duration_s":2.606989,"scheduling_latency_s":2.607003} gitlab # [ 194.196995] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28ff3caf5a2d806b263bb0e6","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28ff3caf5a2d806b263bb0e6: done: 0.009405 sec","job_status":"done","queue_duration_s":2.606989,"scheduling_latency_s":2.607003,"redis_calls":13,"redis_duration_s":0.001581,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009405,"completed_at":"2026-11-10T12:00:10.964Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.198087] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b06433ceb124c27ef87ac08","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b06433ceb124c27ef87ac08: start","job_status":"start","queue_duration_s":2.61727,"scheduling_latency_s":2.617284} gitlab # [ 194.207251] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b06433ceb124c27ef87ac08","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b06433ceb124c27ef87ac08: done: 0.009284 sec","job_status":"done","queue_duration_s":2.61727,"scheduling_latency_s":2.617284,"redis_calls":13,"redis_duration_s":0.001546,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009284,"completed_at":"2026-11-10T12:00:10.975Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.270447] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.976Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92f3275609481ad64b422e6c","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92f3275609481ad64b422e6c: start","job_status":"start","queue_duration_s":2.627419,"scheduling_latency_s":2.627433} gitlab # [ 194.271138] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92f3275609481ad64b422e6c","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92f3275609481ad64b422e6c: done: 0.009608 sec","job_status":"done","queue_duration_s":2.627419,"scheduling_latency_s":2.627433,"redis_calls":13,"redis_duration_s":0.0015350000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009608,"completed_at":"2026-11-10T12:00:10.985Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.325809] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae42c2ec0883599eb745fc10","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae42c2ec0883599eb745fc10: start","job_status":"start","queue_duration_s":2.637914,"scheduling_latency_s":2.637928} gitlab # [ 194.326669] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae42c2ec0883599eb745fc10","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae42c2ec0883599eb745fc10: done: 0.009369 sec","job_status":"done","queue_duration_s":2.637914,"scheduling_latency_s":2.637928,"redis_calls":13,"redis_duration_s":0.001503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009369,"completed_at":"2026-11-10T12:00:10.995Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.366826] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:10.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cc7aa1b7c5442b5e23e6294","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cc7aa1b7c5442b5e23e6294: start","job_status":"start","queue_duration_s":2.648548,"scheduling_latency_s":2.648562} gitlab # [ 194.367763] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cc7aa1b7c5442b5e23e6294","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cc7aa1b7c5442b5e23e6294: done: 0.009501 sec","job_status":"done","queue_duration_s":2.648548,"scheduling_latency_s":2.648562,"redis_calls":13,"redis_duration_s":0.00158,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009501,"completed_at":"2026-11-10T12:00:11.006Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.414144] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"670b74b993fbb22d0af7b585","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-670b74b993fbb22d0af7b585: start","job_status":"start","queue_duration_s":2.659093,"scheduling_latency_s":2.659108} gitlab # [ 194.414899] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"670b74b993fbb22d0af7b585","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-670b74b993fbb22d0af7b585: done: 0.00924 sec","job_status":"done","queue_duration_s":2.659093,"scheduling_latency_s":2.659108,"redis_calls":13,"redis_duration_s":0.001532,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00924,"completed_at":"2026-11-10T12:00:11.016Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.455264] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79f97212b752b7d1c81e0123","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79f97212b752b7d1c81e0123: start","job_status":"start","queue_duration_s":2.669178,"scheduling_latency_s":2.669193} gitlab # [ 194.456069] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.027Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79f97212b752b7d1c81e0123","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79f97212b752b7d1c81e0123: done: 0.00939 sec","job_status":"done","queue_duration_s":2.669178,"scheduling_latency_s":2.669193,"redis_calls":13,"redis_duration_s":0.0015829999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00939,"completed_at":"2026-11-10T12:00:11.027Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.472709] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eabafb4e9b0b32059746bb78","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eabafb4e9b0b32059746bb78: start","job_status":"start","queue_duration_s":2.679431,"scheduling_latency_s":2.679446} gitlab # [ 194.519614] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eabafb4e9b0b32059746bb78","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eabafb4e9b0b32059746bb78: done: 0.009216 sec","job_status":"done","queue_duration_s":2.679431,"scheduling_latency_s":2.679446,"redis_calls":13,"redis_duration_s":0.001494,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001005,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005831,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009216,"completed_at":"2026-11-10T12:00:11.037Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.520600] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.038Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c510466bae7faab1934def00","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c510466bae7faab1934def00: start","job_status":"start","queue_duration_s":2.689516,"scheduling_latency_s":2.68953} gitlab # [ 194.529453] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.047Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c510466bae7faab1934def00","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c510466bae7faab1934def00: done: 0.009178 sec","job_status":"done","queue_duration_s":2.689516,"scheduling_latency_s":2.68953,"redis_calls":13,"redis_duration_s":0.001536,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009178,"completed_at":"2026-11-10T12:00:11.047Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.577305] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c46ff71b031c8278d9eec30","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c46ff71b031c8278d9eec30: start","job_status":"start","queue_duration_s":2.699628,"scheduling_latency_s":2.699642} gitlab # [ 194.578123] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c46ff71b031c8278d9eec30","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c46ff71b031c8278d9eec30: done: 0.009437 sec","job_status":"done","queue_duration_s":2.699628,"scheduling_latency_s":2.699642,"redis_calls":13,"redis_duration_s":0.001562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009437,"completed_at":"2026-11-10T12:00:11.057Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.622820] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05c8f1aa31126c2ce5408d30","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05c8f1aa31126c2ce5408d30: start","job_status":"start","queue_duration_s":2.709954,"scheduling_latency_s":2.709968} gitlab # [ 194.623683] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05c8f1aa31126c2ce5408d30","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05c8f1aa31126c2ce5408d30: done: 0.009252 sec","job_status":"done","queue_duration_s":2.709954,"scheduling_latency_s":2.709968,"redis_calls":13,"redis_duration_s":0.0015639999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009252,"completed_at":"2026-11-10T12:00:11.067Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.683706] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"706ffed848f3a2021dd620d4","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-706ffed848f3a2021dd620d4: start","job_status":"start","queue_duration_s":2.720085,"scheduling_latency_s":2.720099} gitlab # [ 194.684645] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"706ffed848f3a2021dd620d4","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-706ffed848f3a2021dd620d4: done: 0.009368 sec","job_status":"done","queue_duration_s":2.720085,"scheduling_latency_s":2.720099,"redis_calls":13,"redis_duration_s":0.001579,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009368,"completed_at":"2026-11-10T12:00:11.078Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.701654] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"403d5256c563027e3e65d213","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-403d5256c563027e3e65d213: start","job_status":"start","queue_duration_s":2.73034,"scheduling_latency_s":2.730354} gitlab # [ 194.713041] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"403d5256c563027e3e65d213","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-403d5256c563027e3e65d213: done: 0.009286 sec","job_status":"done","queue_duration_s":2.73034,"scheduling_latency_s":2.730354,"redis_calls":13,"redis_duration_s":0.001503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009286,"completed_at":"2026-11-10T12:00:11.088Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.754146] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ffc49c9ae8ddcdf5c1de7f1","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ffc49c9ae8ddcdf5c1de7f1: start","job_status":"start","queue_duration_s":2.740502,"scheduling_latency_s":2.740516} gitlab # [ 194.758671] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ffc49c9ae8ddcdf5c1de7f1","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ffc49c9ae8ddcdf5c1de7f1: done: 0.009272 sec","job_status":"done","queue_duration_s":2.740502,"scheduling_latency_s":2.740516,"redis_calls":13,"redis_duration_s":0.00155,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009272,"completed_at":"2026-11-10T12:00:11.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.801055] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"666b0ecebffb795cbf37ca09","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-666b0ecebffb795cbf37ca09: start","job_status":"start","queue_duration_s":2.750604,"scheduling_latency_s":2.750617} gitlab # [ 194.804211] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"666b0ecebffb795cbf37ca09","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-666b0ecebffb795cbf37ca09: done: 0.013323 sec","job_status":"done","queue_duration_s":2.750604,"scheduling_latency_s":2.750617,"redis_calls":13,"redis_duration_s":0.001836,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001312,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013323,"completed_at":"2026-11-10T12:00:11.112Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.846114] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.113Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fc47f8acd19baa0b2f668cf","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fc47f8acd19baa0b2f668cf: start","job_status":"start","queue_duration_s":2.764965,"scheduling_latency_s":2.76498} gitlab # [ 194.850181] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fc47f8acd19baa0b2f668cf","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fc47f8acd19baa0b2f668cf: done: 0.009357 sec","job_status":"done","queue_duration_s":2.764965,"scheduling_latency_s":2.76498,"redis_calls":13,"redis_duration_s":0.001513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000485,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005847,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009357,"completed_at":"2026-11-10T12:00:11.122Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.916246] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"caef757889817814315eb1d0","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-caef757889817814315eb1d0: start","job_status":"start","queue_duration_s":2.775191,"scheduling_latency_s":2.775206} gitlab # [ 194.916629] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"caef757889817814315eb1d0","created_at":"2026-11-10T12:00:08.105Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-96a98366c33545ea","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"e92e0c9cf09b439b2b4b2b6a28ae585a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.348Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-caef757889817814315eb1d0: done: 0.009252 sec","job_status":"done","queue_duration_s":2.775191,"scheduling_latency_s":2.775206,"redis_calls":13,"redis_duration_s":0.001588,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009252,"completed_at":"2026-11-10T12:00:11.133Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.931962] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.133Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"6497419b655f387af6414976","created_at":"2026-11-10T12:00:08.359Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e09ee4c3651c394ea933539386b6857","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.365Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-6497419b655f387af6414976: start","job_status":"start","queue_duration_s":2.768206,"scheduling_latency_s":2.76822} gitlab # [ 194.943961] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.149Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"6497419b655f387af6414976","created_at":"2026-11-10T12:00:08.359Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e09ee4c3651c394ea933539386b6857","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.365Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-6497419b655f387af6414976: done: 0.015977 sec","job_status":"done","queue_duration_s":2.768206,"scheduling_latency_s":2.76822,"redis_calls":11,"redis_duration_s":0.001402,"redis_read_bytes":10,"redis_write_bytes":1022,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000251,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000767,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000384,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":225,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000215111,"duration_s":0.015977,"completed_at":"2026-11-10T12:00:11.149Z","load_balancing_strategy":"primary","db_duration_s":0.00188,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 194.995108] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.150Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"46169d66a953b6de49344da6","created_at":"2026-11-10T12:00:08.436Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bc14ba2bd72f3ee388b0dd4c616bb055","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.440Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-46169d66a953b6de49344da6: start","job_status":"start","queue_duration_s":2.71067,"scheduling_latency_s":2.710684} gitlab # [ 194.995763] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.155Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 194.996480] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.158Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"46169d66a953b6de49344da6","created_at":"2026-11-10T12:00:08.436Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bc14ba2bd72f3ee388b0dd4c616bb055","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.440Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-46169d66a953b6de49344da6: done: 0.008042 sec","job_status":"done","queue_duration_s":2.71067,"scheduling_latency_s":2.710684,"redis_calls":7,"redis_duration_s":0.00094,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000686,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008042,"completed_at":"2026-11-10T12:00:11.158Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.033274] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.159Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"30d1fcb3a15565ac42a84d7b","created_at":"2026-11-10T12:00:08.461Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d41d28022b9aeb0ebb93406fe898db81","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.479Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-30d1fcb3a15565ac42a84d7b: start","job_status":"start","queue_duration_s":2.68019,"scheduling_latency_s":2.680205} gitlab # [ 195.046596] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.175Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"30d1fcb3a15565ac42a84d7b","created_at":"2026-11-10T12:00:08.461Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d41d28022b9aeb0ebb93406fe898db81","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.479Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-30d1fcb3a15565ac42a84d7b: done: 0.015718 sec","job_status":"done","queue_duration_s":2.68019,"scheduling_latency_s":2.680205,"redis_calls":13,"redis_duration_s":0.001651,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00027,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000665,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009426,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000214274,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.015718,"completed_at":"2026-11-10T12:00:11.175Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001844,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.091723] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.176Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"de04ab6974a54f4685d5bb43","created_at":"2026-11-10T12:00:08.497Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1db07257242f962c9db30d355e21b498","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.502Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-de04ab6974a54f4685d5bb43: start","job_status":"start","queue_duration_s":2.673851,"scheduling_latency_s":2.673864} gitlab # [ 195.133612] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.181Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 195.134681] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.184Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"de04ab6974a54f4685d5bb43","created_at":"2026-11-10T12:00:08.497Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1db07257242f962c9db30d355e21b498","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.502Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-de04ab6974a54f4685d5bb43: done: 0.00796 sec","job_status":"done","queue_duration_s":2.673851,"scheduling_latency_s":2.673864,"redis_calls":7,"redis_duration_s":0.000977,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000681,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000296,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.00796,"completed_at":"2026-11-10T12:00:11.184Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.139715] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.185Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"40c495a2f9d3d8ed5ed5fb82","created_at":"2026-11-10T12:00:08.511Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e84d94c69b7f3260b870abdf3f90936","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T12:00:08.517Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-40c495a2f9d3d8ed5ed5fb82: start","job_status":"start","queue_duration_s":2.66781,"scheduling_latency_s":2.667823} gitlab # [ 195.186475] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.189Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 195.187131] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.192Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"40c495a2f9d3d8ed5ed5fb82","created_at":"2026-11-10T12:00:08.511Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e84d94c69b7f3260b870abdf3f90936","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T12:00:08.517Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-40c495a2f9d3d8ed5ed5fb82: done: 0.007645 sec","job_status":"done","queue_duration_s":2.66781,"scheduling_latency_s":2.667823,"redis_calls":7,"redis_duration_s":0.000895,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000655,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00024,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007645,"completed_at":"2026-11-10T12:00:11.192Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.189070] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.193Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"b7ca624a76763db476a11d1d","created_at":"2026-11-10T12:00:08.562Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c5208bda83e467491d0e51b9daed7ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.576Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-b7ca624a76763db476a11d1d: start","job_status":"start","queue_duration_s":2.617304,"scheduling_latency_s":2.617316} gitlab # [ 195.231172] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.215Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","jid":"b7ca624a76763db476a11d1d","created_at":"2026-11-10T12:00:08.562Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c5208bda83e467491d0e51b9daed7ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fd978bae1387dafc89025ea66dbcf41624e32b5e06562317011e023b1327155","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.576Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleBulkDeleteJobArtifactCronWorker JID-b7ca624a76763db476a11d1d: done: 0.02162 sec","job_status":"done","queue_duration_s":2.617304,"scheduling_latency_s":2.617316,"redis_calls":24,"redis_duration_s":0.0031119999999999997,"redis_read_bytes":25,"redis_write_bytes":5972,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00052,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":704,"redis_queues_calls":2,"redis_queues_duration_s":0.000188,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3805,"redis_queues_metadata_calls":11,"redis_queues_metadata_duration_s":0.001603,"redis_queues_metadata_read_bytes":14,"redis_queues_metadata_write_bytes":894,"redis_shared_state_calls":7,"redis_shared_state_duration_s":0.000801,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":569,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024221,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.max_buckets":5,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.available_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.occupied_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.stale_count":0,"extra.ci_schedule_bulk_delete_job_artifact_cron_worker.missing_count":5,"duration_s":0.02162,"completed_at":"2026-11-10T12:00:11.215Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000573,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.245611] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.216Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"9f2a9cb0e664f60b9faf3209","created_at":"2026-11-10T12:00:08.584Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e56851612d2f79896cd79d5180bd0853","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.596Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-9f2a9cb0e664f60b9faf3209: start","job_status":"start","queue_duration_s":2.619389,"scheduling_latency_s":2.619404} gitlab # [ 195.284508] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.229Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"9f2a9cb0e664f60b9faf3209","created_at":"2026-11-10T12:00:08.584Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e56851612d2f79896cd79d5180bd0853","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.596Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-9f2a9cb0e664f60b9faf3209: done: 0.013006 sec","job_status":"done","queue_duration_s":2.619389,"scheduling_latency_s":2.619404,"redis_calls":10,"redis_duration_s":0.001418,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000813,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000605,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.003313829,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.013006,"completed_at":"2026-11-10T12:00:11.229Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001452,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 195.324843] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.230Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"6883937cdbfb6f0a9cb7aa80","created_at":"2026-11-10T12:00:08.605Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c4d05e6e9a8afd6ebeb7a49d9d8b031","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.609Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-6883937cdbfb6f0a9cb7aa80: start","job_status":"start","queue_duration_s":2.620581,"scheduling_latency_s":2.620595} gitlab # [ 195.325632] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.234Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 195.326126] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.237Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"6883937cdbfb6f0a9cb7aa80","created_at":"2026-11-10T12:00:08.605Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c4d05e6e9a8afd6ebeb7a49d9d8b031","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.609Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-6883937cdbfb6f0a9cb7aa80: done: 0.007617 sec","job_status":"done","queue_duration_s":2.620581,"scheduling_latency_s":2.620595,"redis_calls":7,"redis_duration_s":0.000878,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000637,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004508,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007617,"completed_at":"2026-11-10T12:00:11.237Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.348149] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.238Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"6642c7ea05d56d055f89e0ba","created_at":"2026-11-10T12:00:08.614Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"101db55c828e9d7895f1ac8d949e3746","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.618Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-6642c7ea05d56d055f89e0ba: start","job_status":"start","queue_duration_s":2.620555,"scheduling_latency_s":2.620569} gitlab # [ 195.392638] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.259Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"6642c7ea05d56d055f89e0ba","created_at":"2026-11-10T12:00:08.614Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"101db55c828e9d7895f1ac8d949e3746","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.618Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-6642c7ea05d56d055f89e0ba: done: 0.020371 sec","job_status":"done","queue_duration_s":2.620555,"scheduling_latency_s":2.620569,"redis_calls":9,"redis_duration_s":0.001355,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000693,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000453968,"exclusive_lock_hold_duration_s":0.007908268,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.020371,"completed_at":"2026-11-10T12:00:11.259Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006748,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.399251] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.260Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"53ef5a0a6c53cdff197cbf30","created_at":"2026-11-10T12:00:08.666Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7aacba9ccacb707602a74a06ee0bddb2","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.679Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-53ef5a0a6c53cdff197cbf30: start","job_status":"start","queue_duration_s":2.581062,"scheduling_latency_s":2.581076} gitlab # [ 195.412231] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.272Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"53ef5a0a6c53cdff197cbf30","created_at":"2026-11-10T12:00:08.666Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7aacba9ccacb707602a74a06ee0bddb2","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.679Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-53ef5a0a6c53cdff197cbf30: done: 0.01263 sec","job_status":"done","queue_duration_s":2.581062,"scheduling_latency_s":2.581076,"redis_calls":11,"redis_duration_s":0.001493,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000261,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000797,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00018,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000198908,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.01263,"completed_at":"2026-11-10T12:00:11.272Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000298,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.489161] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.273Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ff06285bb57b2a14b9ea6a65","created_at":"2026-11-10T12:00:08.698Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4feda4b37325dd654d84496e53f8a84a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.704Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ff06285bb57b2a14b9ea6a65: start","job_status":"start","queue_duration_s":2.56962,"scheduling_latency_s":2.569634} gitlab # [ 195.489984] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.391Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ff06285bb57b2a14b9ea6a65","created_at":"2026-11-10T12:00:08.698Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4feda4b37325dd654d84496e53f8a84a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.704Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ff06285bb57b2a14b9ea6a65: done: 0.1181 sec","job_status":"done","queue_duration_s":2.56962,"scheduling_latency_s":2.569634,"redis_calls":11,"redis_duration_s":0.001712,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000699,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000755,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.096618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000335238,"exclusive_lock_hold_duration_s":0.106599303,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.1181,"completed_at":"2026-11-10T12:00:11.391Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02669,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.530605] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.392Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"f07fca3dd23a43cc19e812a9","created_at":"2026-11-10T12:00:08.722Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69b2a8b5c53eacc89219de0dc228743c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T12:00:08.727Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-f07fca3dd23a43cc19e812a9: start","job_status":"start","queue_duration_s":2.665652,"scheduling_latency_s":2.665668} gitlab # [ 195.531182] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.398Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 195.531717] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.401Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"f07fca3dd23a43cc19e812a9","created_at":"2026-11-10T12:00:08.722Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-e6973ea9f619495b","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69b2a8b5c53eacc89219de0dc228743c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T12:00:08.727Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-f07fca3dd23a43cc19e812a9: done: 0.008495 sec","job_status":"done","queue_duration_s":2.665652,"scheduling_latency_s":2.665668,"redis_calls":7,"redis_duration_s":0.0009500000000000001,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000711,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000239,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008495,"completed_at":"2026-11-10T12:00:11.401Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.577966] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.402Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"aba94416a7871df4396eaf82","created_at":"2026-11-10T12:00:08.925Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-7946bd9dbea9422d","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"319d47dcfe64f4133f783fda449cdb4e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.939Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-aba94416a7871df4396eaf82: start","job_status":"start","queue_duration_s":2.463194,"scheduling_latency_s":2.463208} gitlab # [ 195.578678] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.423Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"aba94416a7871df4396eaf82","created_at":"2026-11-10T12:00:08.925Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-7946bd9dbea9422d","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"319d47dcfe64f4133f783fda449cdb4e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.939Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-aba94416a7871df4396eaf82: done: 0.021428 sec","job_status":"done","queue_duration_s":2.463194,"scheduling_latency_s":2.463208,"redis_calls":10,"redis_duration_s":0.00183,"redis_read_bytes":9,"redis_write_bytes":1117,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000299,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001106,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000425,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":222,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000291099,"duration_s":0.021428,"completed_at":"2026-11-10T12:00:11.423Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001581,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.632384] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.456Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"0449c5209d4c82b16da9380c","created_at":"2026-11-10T12:00:08.940Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-7946bd9dbea9422d","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"319d47dcfe64f4133f783fda449cdb4e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.945Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-0449c5209d4c82b16da9380c: start","job_status":"start","queue_duration_s":2.510574,"scheduling_latency_s":2.510599} gitlab # [ 195.633109] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.475Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"0449c5209d4c82b16da9380c","created_at":"2026-11-10T12:00:08.940Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-7946bd9dbea9422d","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"319d47dcfe64f4133f783fda449cdb4e","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:08.945Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-0449c5209d4c82b16da9380c: done: 0.019337 sec","job_status":"done","queue_duration_s":2.510574,"scheduling_latency_s":2.510599,"redis_calls":10,"redis_duration_s":0.002114,"redis_read_bytes":9,"redis_write_bytes":1133,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000271,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001005,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000838,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000241372,"duration_s":0.019337,"completed_at":"2026-11-10T12:00:11.475Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000811,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.655949] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.477Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8cabbc2d3922f627b91e138d","created_at":"2026-11-10T12:00:09.190Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-29cb3b59f0564589","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8d6923920d1a1fd2a69b150163ad4fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:09.201Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8cabbc2d3922f627b91e138d: start","job_status":"start","queue_duration_s":2.275768,"scheduling_latency_s":2.275795} gitlab # [ 195.667101] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.484Z","message":"No existing merge request to be cleaned up."} gitlab # [ 195.668691] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.490Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8cabbc2d3922f627b91e138d","created_at":"2026-11-10T12:00:09.190Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-29cb3b59f0564589","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8d6923920d1a1fd2a69b150163ad4fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:09.201Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8cabbc2d3922f627b91e138d: done: 0.013013 sec","job_status":"done","queue_duration_s":2.275768,"scheduling_latency_s":2.275795,"redis_calls":10,"redis_duration_s":0.0013189999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000792,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013013,"completed_at":"2026-11-10T12:00:11.490Z","load_balancing_strategy":"primary","db_duration_s":0.001275,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.709855] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.495Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8befa5ac2023c0732be88eb4","created_at":"2026-11-10T12:00:09.190Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-29cb3b59f0564589","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8d6923920d1a1fd2a69b150163ad4fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:09.201Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8befa5ac2023c0732be88eb4: start","job_status":"start","queue_duration_s":2.293747,"scheduling_latency_s":2.293762} gitlab # [ 195.713547] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.502Z","message":"No existing merge request to be cleaned up."} gitlab # [ 195.715064] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.507Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8befa5ac2023c0732be88eb4","created_at":"2026-11-10T12:00:09.190Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-29cb3b59f0564589","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8d6923920d1a1fd2a69b150163ad4fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:09.201Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8befa5ac2023c0732be88eb4: done: 0.01182 sec","job_status":"done","queue_duration_s":2.293747,"scheduling_latency_s":2.293762,"redis_calls":10,"redis_duration_s":0.0012460000000000001,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000727,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01182,"completed_at":"2026-11-10T12:00:11.507Z","load_balancing_strategy":"primary","db_duration_s":0.001048,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.771783] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.508Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"38dfb56c7caa8cc891b53fc5","created_at":"2026-11-10T12:00:09.190Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-29cb3b59f0564589","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8d6923920d1a1fd2a69b150163ad4fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:09.201Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-38dfb56c7caa8cc891b53fc5: start","job_status":"start","queue_duration_s":2.306398,"scheduling_latency_s":2.306438} gitlab # [ 195.772661] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.515Z","message":"No existing merge request to be cleaned up."} gitlab # [ 195.773234] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.520Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"38dfb56c7caa8cc891b53fc5","created_at":"2026-11-10T12:00:09.190Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-29cb3b59f0564589","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8d6923920d1a1fd2a69b150163ad4fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:09.201Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-38dfb56c7caa8cc891b53fc5: done: 0.012198 sec","job_status":"done","queue_duration_s":2.306398,"scheduling_latency_s":2.306438,"redis_calls":10,"redis_duration_s":0.001256,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000739,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012198,"completed_at":"2026-11-10T12:00:11.520Z","load_balancing_strategy":"primary","db_duration_s":0.001024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.819906] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.521Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"eedc9bcc89e74ab312a7287d","created_at":"2026-11-10T12:00:09.190Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-29cb3b59f0564589","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8d6923920d1a1fd2a69b150163ad4fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:09.201Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-eedc9bcc89e74ab312a7287d: start","job_status":"start","queue_duration_s":2.319482,"scheduling_latency_s":2.319496} gitlab # [ 195.820526] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.527Z","message":"No existing merge request to be cleaned up."} gitlab # [ 195.821487] postgres[2530]: [2530] ERROR: relation "table_without_models" does not exist gitlab # [ 195.822203] postgres[2530]: [2530] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 195.822718] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.532Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"eedc9bcc89e74ab312a7287d","created_at":"2026-11-10T12:00:09.190Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-29cb3b59f0564589","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e8d6923920d1a1fd2a69b150163ad4fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:09.201Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-eedc9bcc89e74ab312a7287d: done: 0.0114 sec","job_status":"done","queue_duration_s":2.319482,"scheduling_latency_s":2.319496,"redis_calls":10,"redis_duration_s":0.00123,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00071,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0114,"completed_at":"2026-11-10T12:00:11.532Z","load_balancing_strategy":"primary","db_duration_s":0.000997,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.871662] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"3fca3f6bddd2e9275b330939","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-3fca3f6bddd2e9275b330939: start","job_status":"start","queue_duration_s":1.03957,"scheduling_latency_s":1.039586} gitlab # [ 195.872514] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"3fca3f6bddd2e9275b330939","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-3fca3f6bddd2e9275b330939: done: 0.039836 sec","job_status":"done","queue_duration_s":1.03957,"scheduling_latency_s":1.039586,"redis_calls":16,"redis_duration_s":0.0073,"redis_read_bytes":14,"redis_write_bytes":1526,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0018,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001708,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":423,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003792,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":802,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00055454,"duration_s":0.039836,"completed_at":"2026-11-10T12:00:11.575Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011366,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.922582] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.576Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"5c8baf76f7c000c039a11785","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-5c8baf76f7c000c039a11785: start","job_status":"start","queue_duration_s":1.080403,"scheduling_latency_s":1.080445} gitlab # [ 195.923489] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.587Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"5c8baf76f7c000c039a11785","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-5c8baf76f7c000c039a11785: done: 0.010855 sec","job_status":"done","queue_duration_s":1.080403,"scheduling_latency_s":1.080445,"redis_calls":13,"redis_duration_s":0.0016120000000000002,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010855,"completed_at":"2026-11-10T12:00:11.587Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000704,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.939223] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"df2ae7d0fde7efacf5ecfabc","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-df2ae7d0fde7efacf5ecfabc: start","job_status":"start","queue_duration_s":1.092145,"scheduling_latency_s":1.09216} gitlab # [ 195.980833] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"df2ae7d0fde7efacf5ecfabc","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-df2ae7d0fde7efacf5ecfabc: done: 0.010871 sec","job_status":"done","queue_duration_s":1.092145,"scheduling_latency_s":1.09216,"redis_calls":13,"redis_duration_s":0.001595,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001082,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010871,"completed_at":"2026-11-10T12:00:11.598Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000632,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.985463] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"63f285d3ca11ff8e6a89318c","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-63f285d3ca11ff8e6a89318c: start","job_status":"start","queue_duration_s":1.10438,"scheduling_latency_s":1.104394} gitlab # [ 196.031246] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"63f285d3ca11ff8e6a89318c","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-63f285d3ca11ff8e6a89318c: done: 0.011194 sec","job_status":"done","queue_duration_s":1.10438,"scheduling_latency_s":1.104394,"redis_calls":13,"redis_duration_s":0.001679,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011194,"completed_at":"2026-11-10T12:00:11.611Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000616,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.032243] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"a428c8b7524aee0d9359f445","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-a428c8b7524aee0d9359f445: start","job_status":"start","queue_duration_s":1.116485,"scheduling_latency_s":1.116499} gitlab # [ 196.043538] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.623Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"a428c8b7524aee0d9359f445","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-a428c8b7524aee0d9359f445: done: 0.010948 sec","job_status":"done","queue_duration_s":1.116485,"scheduling_latency_s":1.116499,"redis_calls":13,"redis_duration_s":0.001592,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006917,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010948,"completed_at":"2026-11-10T12:00:11.623Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000616,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.093084] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"03fc494a0388a951984530ae","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-03fc494a0388a951984530ae: start","job_status":"start","queue_duration_s":1.128318,"scheduling_latency_s":1.128333} gitlab # [ 196.093640] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"03fc494a0388a951984530ae","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-03fc494a0388a951984530ae: done: 0.010911 sec","job_status":"done","queue_duration_s":1.128318,"scheduling_latency_s":1.128333,"redis_calls":13,"redis_duration_s":0.0016810000000000002,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001128,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010911,"completed_at":"2026-11-10T12:00:11.635Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000641,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.141522] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"447603ee964cda991d5eb110","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-447603ee964cda991d5eb110: start","job_status":"start","queue_duration_s":1.140146,"scheduling_latency_s":1.14016} gitlab # [ 196.142258] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.646Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"447603ee964cda991d5eb110","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-447603ee964cda991d5eb110: done: 0.010723 sec","job_status":"done","queue_duration_s":1.140146,"scheduling_latency_s":1.14016,"redis_calls":13,"redis_duration_s":0.001788,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006264,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010723,"completed_at":"2026-11-10T12:00:11.646Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000448,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.180562] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"22b6e551a96fd8d16b302f74","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-22b6e551a96fd8d16b302f74: start","job_status":"start","queue_duration_s":1.152246,"scheduling_latency_s":1.152261} gitlab # [ 196.182470] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"22b6e551a96fd8d16b302f74","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-22b6e551a96fd8d16b302f74: done: 0.010722 sec","job_status":"done","queue_duration_s":1.152246,"scheduling_latency_s":1.152261,"redis_calls":13,"redis_duration_s":0.001532,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000481,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010722,"completed_at":"2026-11-10T12:00:11.658Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000672,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.220789] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"73085fb45e53dfbdd8d172a9","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-73085fb45e53dfbdd8d172a9: start","job_status":"start","queue_duration_s":1.163832,"scheduling_latency_s":1.163847} gitlab # [ 196.261578] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"73085fb45e53dfbdd8d172a9","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-73085fb45e53dfbdd8d172a9: done: 0.010183 sec","job_status":"done","queue_duration_s":1.163832,"scheduling_latency_s":1.163847,"redis_calls":13,"redis_duration_s":0.001496,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000477,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010183,"completed_at":"2026-11-10T12:00:11.669Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00038,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.262911] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.670Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"2917cc065a3b734f5f9d300b","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-2917cc065a3b734f5f9d300b: start","job_status":"start","queue_duration_s":1.174867,"scheduling_latency_s":1.174881} gitlab # [ 196.309673] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"2917cc065a3b734f5f9d300b","created_at":"2026-11-10T12:00:10.484Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-45debef273aa4ef6","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"661d806db8d87dc1e6f41b71b4c2e147","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.495Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-2917cc065a3b734f5f9d300b: done: 0.010312 sec","job_status":"done","queue_duration_s":1.174867,"scheduling_latency_s":1.174881,"redis_calls":13,"redis_duration_s":0.0015600000000000002,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010312,"completed_at":"2026-11-10T12:00:11.681Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.310805] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.687Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"95c0fbff121b327ceee67dd6","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-95c0fbff121b327ceee67dd6: start","job_status":"start","queue_duration_s":1.104947,"scheduling_latency_s":1.104963} gitlab # [ 196.354741] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"95c0fbff121b327ceee67dd6","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-95c0fbff121b327ceee67dd6: done: 0.020593 sec","job_status":"done","queue_duration_s":1.104947,"scheduling_latency_s":1.104963,"redis_calls":13,"redis_duration_s":0.001918,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000264,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000637,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212876,"duration_s":0.020593,"completed_at":"2026-11-10T12:00:11.708Z","load_balancing_strategy":"primary","db_duration_s":0.002779,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.355812] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.709Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5321510b6cd137d4de20ad63","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5321510b6cd137d4de20ad63: start","job_status":"start","queue_duration_s":1.126425,"scheduling_latency_s":1.126438} gitlab # [ 196.366467] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5321510b6cd137d4de20ad63","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5321510b6cd137d4de20ad63: done: 0.011592 sec","job_status":"done","queue_duration_s":1.126425,"scheduling_latency_s":1.126438,"redis_calls":10,"redis_duration_s":0.0012599999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000734,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011592,"completed_at":"2026-11-10T12:00:11.720Z","load_balancing_strategy":"primary","db_duration_s":0.001049,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.401604] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"154e01433027c5bd8ab2f0e5","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-154e01433027c5bd8ab2f0e5: start","job_status":"start","queue_duration_s":1.138875,"scheduling_latency_s":1.13889} gitlab # [ 196.412629] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"154e01433027c5bd8ab2f0e5","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-154e01433027c5bd8ab2f0e5: done: 0.016265 sec","job_status":"done","queue_duration_s":1.138875,"scheduling_latency_s":1.13889,"redis_calls":10,"redis_duration_s":0.001258,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000738,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016265,"completed_at":"2026-11-10T12:00:11.737Z","load_balancing_strategy":"primary","db_duration_s":0.00843,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.462640] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7770b235893766655d1d8d72","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7770b235893766655d1d8d72: start","job_status":"start","queue_duration_s":1.156022,"scheduling_latency_s":1.156037} gitlab # [ 196.463556] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7770b235893766655d1d8d72","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7770b235893766655d1d8d72: done: 0.011464 sec","job_status":"done","queue_duration_s":1.156022,"scheduling_latency_s":1.156037,"redis_calls":10,"redis_duration_s":0.0012360000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000735,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007896,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011464,"completed_at":"2026-11-10T12:00:11.750Z","load_balancing_strategy":"primary","db_duration_s":0.001045,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.507195] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6bd49755c6dc21b7ee50d8a9","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6bd49755c6dc21b7ee50d8a9: start","job_status":"start","queue_duration_s":1.168333,"scheduling_latency_s":1.168347} gitlab # [ 196.507890] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.762Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6bd49755c6dc21b7ee50d8a9","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6bd49755c6dc21b7ee50d8a9: done: 0.011279 sec","job_status":"done","queue_duration_s":1.168333,"scheduling_latency_s":1.168347,"redis_calls":10,"redis_duration_s":0.001229,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000732,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007741,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011279,"completed_at":"2026-11-10T12:00:11.762Z","load_balancing_strategy":"primary","db_duration_s":0.000968,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.549138] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2116cc30562063c6b5e3312d","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2116cc30562063c6b5e3312d: start","job_status":"start","queue_duration_s":1.180971,"scheduling_latency_s":1.180986} gitlab # [ 196.549793] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2116cc30562063c6b5e3312d","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2116cc30562063c6b5e3312d: done: 0.022328 sec","job_status":"done","queue_duration_s":1.180971,"scheduling_latency_s":1.180986,"redis_calls":10,"redis_duration_s":0.0057859999999999995,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00139,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004396,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022328,"completed_at":"2026-11-10T12:00:11.785Z","load_balancing_strategy":"primary","db_duration_s":0.010549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.614397] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b6877170b165683dda4c019e","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b6877170b165683dda4c019e: start","job_status":"start","queue_duration_s":1.205331,"scheduling_latency_s":1.205347} gitlab # [ 196.615236] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b6877170b165683dda4c019e","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b6877170b165683dda4c019e: done: 0.011712 sec","job_status":"done","queue_duration_s":1.205331,"scheduling_latency_s":1.205347,"redis_calls":10,"redis_duration_s":0.0012690000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000738,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011712,"completed_at":"2026-11-10T12:00:11.799Z","load_balancing_strategy":"primary","db_duration_s":0.001128,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.624741] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.801Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"03999702be4ed518ab28d839","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-03999702be4ed518ab28d839: start","job_status":"start","queue_duration_s":1.218526,"scheduling_latency_s":1.218541} gitlab # [ 196.664435] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"03999702be4ed518ab28d839","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-03999702be4ed518ab28d839: done: 0.011679 sec","job_status":"done","queue_duration_s":1.218526,"scheduling_latency_s":1.218541,"redis_calls":10,"redis_duration_s":0.001218,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000703,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011679,"completed_at":"2026-11-10T12:00:11.812Z","load_balancing_strategy":"primary","db_duration_s":0.001007,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.669267] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.813Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0642fe47cbb4ca03d468cff8","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0642fe47cbb4ca03d468cff8: start","job_status":"start","queue_duration_s":1.231059,"scheduling_latency_s":1.231073} gitlab # [ 196.680293] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.825Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0642fe47cbb4ca03d468cff8","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0642fe47cbb4ca03d468cff8: done: 0.011445 sec","job_status":"done","queue_duration_s":1.231059,"scheduling_latency_s":1.231073,"redis_calls":10,"redis_duration_s":0.00166,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001145,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011445,"completed_at":"2026-11-10T12:00:11.825Z","load_balancing_strategy":"primary","db_duration_s":0.000986,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.730635] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"df430f283449e094ba658899","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-df430f283449e094ba658899: start","job_status":"start","queue_duration_s":1.24648,"scheduling_latency_s":1.246495} gitlab # [ 196.731251] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"df430f283449e094ba658899","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-df430f283449e094ba658899: done: 0.014871 sec","job_status":"done","queue_duration_s":1.24648,"scheduling_latency_s":1.246495,"redis_calls":10,"redis_duration_s":0.002647,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001883,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008833,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014871,"completed_at":"2026-11-10T12:00:11.843Z","load_balancing_strategy":"primary","db_duration_s":0.001568,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.783591] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9d7f0fc49e0fd183d380c8fc","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9d7f0fc49e0fd183d380c8fc: start","job_status":"start","queue_duration_s":1.262282,"scheduling_latency_s":1.262297} gitlab # [ 196.784273] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9d7f0fc49e0fd183d380c8fc","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9d7f0fc49e0fd183d380c8fc: done: 0.011142 sec","job_status":"done","queue_duration_s":1.262282,"scheduling_latency_s":1.262297,"redis_calls":10,"redis_duration_s":0.001303,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000773,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011142,"completed_at":"2026-11-10T12:00:11.856Z","load_balancing_strategy":"primary","db_duration_s":0.001048,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.803890] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3690684a90f45bdb0cf278a9","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3690684a90f45bdb0cf278a9: start","job_status":"start","queue_duration_s":1.274315,"scheduling_latency_s":1.27433} gitlab # [ 196.848413] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3690684a90f45bdb0cf278a9","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3690684a90f45bdb0cf278a9: done: 0.015998 sec","job_status":"done","queue_duration_s":1.274315,"scheduling_latency_s":1.27433,"redis_calls":10,"redis_duration_s":0.005081,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000593,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004488,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015998,"completed_at":"2026-11-10T12:00:11.872Z","load_balancing_strategy":"primary","db_duration_s":0.001095,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.849425] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d4fc185c69a1fd9e2e3bc396","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d4fc185c69a1fd9e2e3bc396: start","job_status":"start","queue_duration_s":1.291255,"scheduling_latency_s":1.291271} gitlab # [ 196.859652] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.885Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d4fc185c69a1fd9e2e3bc396","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d4fc185c69a1fd9e2e3bc396: done: 0.01201 sec","job_status":"done","queue_duration_s":1.291255,"scheduling_latency_s":1.291271,"redis_calls":10,"redis_duration_s":0.001312,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00078,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01201,"completed_at":"2026-11-10T12:00:11.885Z","load_balancing_strategy":"primary","db_duration_s":0.001166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.910866] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.886Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b9a9551d6128bde0cae416ac","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b9a9551d6128bde0cae416ac: start","job_status":"start","queue_duration_s":1.304154,"scheduling_latency_s":1.304168} gitlab # [ 196.911771] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.902Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b9a9551d6128bde0cae416ac","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b9a9551d6128bde0cae416ac: done: 0.015937 sec","job_status":"done","queue_duration_s":1.304154,"scheduling_latency_s":1.304168,"redis_calls":10,"redis_duration_s":0.004635,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004065,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015937,"completed_at":"2026-11-10T12:00:11.902Z","load_balancing_strategy":"primary","db_duration_s":0.001072,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 196.938221] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"62bb91ff807360245206bc65","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-62bb91ff807360245206bc65: start","job_status":"start","queue_duration_s":1.321008,"scheduling_latency_s":1.321034} gitlab # [ 196.949440] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"62bb91ff807360245206bc65","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-62bb91ff807360245206bc65: done: 0.016399 sec","job_status":"done","queue_duration_s":1.321008,"scheduling_latency_s":1.321034,"redis_calls":10,"redis_duration_s":0.005945,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003672,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002273,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008509,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016399,"completed_at":"2026-11-10T12:00:11.920Z","load_balancing_strategy":"primary","db_duration_s":0.00149,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.002313] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f35f19ebe0529e895c28a1e6","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f35f19ebe0529e895c28a1e6: start","job_status":"start","queue_duration_s":1.338424,"scheduling_latency_s":1.338439} gitlab # [ 197.002997] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f35f19ebe0529e895c28a1e6","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f35f19ebe0529e895c28a1e6: done: 0.011947 sec","job_status":"done","queue_duration_s":1.338424,"scheduling_latency_s":1.338439,"redis_calls":10,"redis_duration_s":0.001389,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000827,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011947,"completed_at":"2026-11-10T12:00:11.933Z","load_balancing_strategy":"primary","db_duration_s":0.001065,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.044995] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6876b4563ede823e05252d58","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6876b4563ede823e05252d58: start","job_status":"start","queue_duration_s":1.351258,"scheduling_latency_s":1.351272} gitlab # [ 197.045744] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6876b4563ede823e05252d58","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6876b4563ede823e05252d58: done: 0.012536 sec","job_status":"done","queue_duration_s":1.351258,"scheduling_latency_s":1.351272,"redis_calls":10,"redis_duration_s":0.001323,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000763,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012536,"completed_at":"2026-11-10T12:00:11.946Z","load_balancing_strategy":"primary","db_duration_s":0.001076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.105993] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cb21401e035ad8c0ec616cda","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cb21401e035ad8c0ec616cda: start","job_status":"start","queue_duration_s":1.364684,"scheduling_latency_s":1.364699} gitlab # [ 197.106800] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cb21401e035ad8c0ec616cda","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cb21401e035ad8c0ec616cda: done: 0.022709 sec","job_status":"done","queue_duration_s":1.364684,"scheduling_latency_s":1.364699,"redis_calls":10,"redis_duration_s":0.007543,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002431,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.005112,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022709,"completed_at":"2026-11-10T12:00:11.970Z","load_balancing_strategy":"primary","db_duration_s":0.00644,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.117198] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"334222d2b384eb01b5fd33bf","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-334222d2b384eb01b5fd33bf: start","job_status":"start","queue_duration_s":1.388292,"scheduling_latency_s":1.388308} gitlab # [ 197.128121] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"334222d2b384eb01b5fd33bf","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-334222d2b384eb01b5fd33bf: done: 0.011442 sec","job_status":"done","queue_duration_s":1.388292,"scheduling_latency_s":1.388308,"redis_calls":10,"redis_duration_s":0.001376,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000839,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011442,"completed_at":"2026-11-10T12:00:11.982Z","load_balancing_strategy":"primary","db_duration_s":0.001047,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.187073] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.983Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4aa65567abb090b8d0e411a4","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4aa65567abb090b8d0e411a4: start","job_status":"start","queue_duration_s":1.40076,"scheduling_latency_s":1.400775} gitlab # [ 197.187682] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4aa65567abb090b8d0e411a4","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4aa65567abb090b8d0e411a4: done: 0.011948 sec","job_status":"done","queue_duration_s":1.40076,"scheduling_latency_s":1.400775,"redis_calls":10,"redis_duration_s":0.0014629999999999999,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000896,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007587,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011948,"completed_at":"2026-11-10T12:00:11.995Z","load_balancing_strategy":"primary","db_duration_s":0.001081,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.240717] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:11.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"25196114086dbb013537ccf2","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-25196114086dbb013537ccf2: start","job_status":"start","queue_duration_s":1.413597,"scheduling_latency_s":1.413612} gitlab # [ 197.241465] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"25196114086dbb013537ccf2","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-25196114086dbb013537ccf2: done: 0.015631 sec","job_status":"done","queue_duration_s":1.413597,"scheduling_latency_s":1.413612,"redis_calls":10,"redis_duration_s":0.00502,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003742,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001278,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015631,"completed_at":"2026-11-10T12:00:12.011Z","load_balancing_strategy":"primary","db_duration_s":0.001101,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.252045] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.014Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9b6f55b36f25b3da842a5d3c","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9b6f55b36f25b3da842a5d3c: start","job_status":"start","queue_duration_s":1.432189,"scheduling_latency_s":1.432204} gitlab # [ 197.263132] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9b6f55b36f25b3da842a5d3c","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9b6f55b36f25b3da842a5d3c: done: 0.019042 sec","job_status":"done","queue_duration_s":1.432189,"scheduling_latency_s":1.432204,"redis_calls":10,"redis_duration_s":0.005655,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003787,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019042,"completed_at":"2026-11-10T12:00:12.033Z","load_balancing_strategy":"primary","db_duration_s":0.004161,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.307964] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6c8443a179ce79a252daaf2c","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6c8443a179ce79a252daaf2c: start","job_status":"start","queue_duration_s":1.453856,"scheduling_latency_s":1.453872} gitlab # [ 197.308732] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6c8443a179ce79a252daaf2c","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6c8443a179ce79a252daaf2c: done: 0.014223 sec","job_status":"done","queue_duration_s":1.453856,"scheduling_latency_s":1.453872,"redis_calls":10,"redis_duration_s":0.002761,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014223,"completed_at":"2026-11-10T12:00:12.050Z","load_balancing_strategy":"primary","db_duration_s":0.001108,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.348981] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.051Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9a9e814e4b8d378097b5a001","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9a9e814e4b8d378097b5a001: start","job_status":"start","queue_duration_s":1.469057,"scheduling_latency_s":1.469071} gitlab # [ 197.352566] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9a9e814e4b8d378097b5a001","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9a9e814e4b8d378097b5a001: done: 0.011552 sec","job_status":"done","queue_duration_s":1.469057,"scheduling_latency_s":1.469071,"redis_calls":10,"redis_duration_s":0.001377,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00082,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011552,"completed_at":"2026-11-10T12:00:12.063Z","load_balancing_strategy":"primary","db_duration_s":0.001052,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.407202] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.064Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6578b2bd13ceb78ee29da22f","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6578b2bd13ceb78ee29da22f: start","job_status":"start","queue_duration_s":1.481483,"scheduling_latency_s":1.481497} gitlab # [ 197.407940] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6578b2bd13ceb78ee29da22f","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6578b2bd13ceb78ee29da22f: done: 0.011469 sec","job_status":"done","queue_duration_s":1.481483,"scheduling_latency_s":1.481497,"redis_calls":10,"redis_duration_s":0.001337,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000785,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011469,"completed_at":"2026-11-10T12:00:12.075Z","load_balancing_strategy":"primary","db_duration_s":0.001056,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.453115] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"386628b7ae4a40e777211f7e","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-386628b7ae4a40e777211f7e: start","job_status":"start","queue_duration_s":1.493838,"scheduling_latency_s":1.493852} gitlab # [ 197.453873] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.096Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"386628b7ae4a40e777211f7e","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-386628b7ae4a40e777211f7e: done: 0.020491 sec","job_status":"done","queue_duration_s":1.493838,"scheduling_latency_s":1.493852,"redis_calls":10,"redis_duration_s":0.007732999999999999,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001449,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.006284,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020491,"completed_at":"2026-11-10T12:00:12.096Z","load_balancing_strategy":"primary","db_duration_s":0.003283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.494058] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8370a3926a7eb1ccb6197b69","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8370a3926a7eb1ccb6197b69: start","job_status":"start","queue_duration_s":1.515255,"scheduling_latency_s":1.51527} gitlab # [ 197.494760] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8370a3926a7eb1ccb6197b69","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8370a3926a7eb1ccb6197b69: done: 0.01153 sec","job_status":"done","queue_duration_s":1.515255,"scheduling_latency_s":1.51527,"redis_calls":10,"redis_duration_s":0.001314,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000786,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01153,"completed_at":"2026-11-10T12:00:12.109Z","load_balancing_strategy":"primary","db_duration_s":0.001042,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.543335] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"576f11bb6fa376377652e26c","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-576f11bb6fa376377652e26c: start","job_status":"start","queue_duration_s":1.52763,"scheduling_latency_s":1.527644} gitlab # [ 197.544030] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.121Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"576f11bb6fa376377652e26c","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-576f11bb6fa376377652e26c: done: 0.011692 sec","job_status":"done","queue_duration_s":1.52763,"scheduling_latency_s":1.527644,"redis_calls":10,"redis_duration_s":0.001532,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007547,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011692,"completed_at":"2026-11-10T12:00:12.121Z","load_balancing_strategy":"primary","db_duration_s":0.001062,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.584117] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"22334befd1690ee55fd3a1fd","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-22334befd1690ee55fd3a1fd: start","job_status":"start","queue_duration_s":1.540501,"scheduling_latency_s":1.540516} gitlab # [ 197.584853] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"22334befd1690ee55fd3a1fd","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-22334befd1690ee55fd3a1fd: done: 0.015211 sec","job_status":"done","queue_duration_s":1.540501,"scheduling_latency_s":1.540516,"redis_calls":10,"redis_duration_s":0.004838,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003631,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001207,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015211,"completed_at":"2026-11-10T12:00:12.138Z","load_balancing_strategy":"primary","db_duration_s":0.001098,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.642608] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"efb886923ad860bf0e287173","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-efb886923ad860bf0e287173: start","job_status":"start","queue_duration_s":1.55689,"scheduling_latency_s":1.556904} gitlab # [ 197.643492] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"efb886923ad860bf0e287173","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-efb886923ad860bf0e287173: done: 0.011612 sec","job_status":"done","queue_duration_s":1.55689,"scheduling_latency_s":1.556904,"redis_calls":10,"redis_duration_s":0.001278,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000751,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011612,"completed_at":"2026-11-10T12:00:12.151Z","load_balancing_strategy":"primary","db_duration_s":0.001074,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.692263] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5da025f53e53311dcda5356b","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5da025f53e53311dcda5356b: start","job_status":"start","queue_duration_s":1.569412,"scheduling_latency_s":1.569427} gitlab # [ 197.693060] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5da025f53e53311dcda5356b","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5da025f53e53311dcda5356b: done: 0.011379 sec","job_status":"done","queue_duration_s":1.569412,"scheduling_latency_s":1.569427,"redis_calls":10,"redis_duration_s":0.00131,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000777,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011379,"completed_at":"2026-11-10T12:00:12.163Z","load_balancing_strategy":"primary","db_duration_s":0.001025,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.701824] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.164Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"757edde2d5c822795fa9b343","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-757edde2d5c822795fa9b343: start","job_status":"start","queue_duration_s":1.581792,"scheduling_latency_s":1.581806} gitlab # [ 197.741643] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.180Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"757edde2d5c822795fa9b343","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-757edde2d5c822795fa9b343: done: 0.015805 sec","job_status":"done","queue_duration_s":1.581792,"scheduling_latency_s":1.581806,"redis_calls":10,"redis_duration_s":0.005327,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001375,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015805,"completed_at":"2026-11-10T12:00:12.180Z","load_balancing_strategy":"primary","db_duration_s":0.001036,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.746149] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fbcfb49019db52424a022a46","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fbcfb49019db52424a022a46: start","job_status":"start","queue_duration_s":1.598576,"scheduling_latency_s":1.598591} gitlab # [ 197.757560] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fbcfb49019db52424a022a46","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fbcfb49019db52424a022a46: done: 0.011573 sec","job_status":"done","queue_duration_s":1.598576,"scheduling_latency_s":1.598591,"redis_calls":10,"redis_duration_s":0.0013779999999999999,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000597,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000781,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007742,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011573,"completed_at":"2026-11-10T12:00:12.192Z","load_balancing_strategy":"primary","db_duration_s":0.001163,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.804069] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f5b011fdbdf1c171275228fc","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f5b011fdbdf1c171275228fc: start","job_status":"start","queue_duration_s":1.611043,"scheduling_latency_s":1.611058} gitlab # [ 197.804799] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.206Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f5b011fdbdf1c171275228fc","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f5b011fdbdf1c171275228fc: done: 0.012308 sec","job_status":"done","queue_duration_s":1.611043,"scheduling_latency_s":1.611058,"redis_calls":10,"redis_duration_s":0.0013080000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000772,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012308,"completed_at":"2026-11-10T12:00:12.205Z","load_balancing_strategy":"primary","db_duration_s":0.001056,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.863350] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.206Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0dad2d70a922e5cc9bc0e8b6","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0dad2d70a922e5cc9bc0e8b6: start","job_status":"start","queue_duration_s":1.624249,"scheduling_latency_s":1.624263} gitlab # [ 197.864055] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0dad2d70a922e5cc9bc0e8b6","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0dad2d70a922e5cc9bc0e8b6: done: 0.015132 sec","job_status":"done","queue_duration_s":1.624249,"scheduling_latency_s":1.624263,"redis_calls":10,"redis_duration_s":0.004998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004202,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000796,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015132,"completed_at":"2026-11-10T12:00:12.222Z","load_balancing_strategy":"primary","db_duration_s":0.001014,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.913665] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9957ea3f43dd6b61f2969ebf","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9957ea3f43dd6b61f2969ebf: start","job_status":"start","queue_duration_s":1.641907,"scheduling_latency_s":1.641923} gitlab # [ 197.914568] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9957ea3f43dd6b61f2969ebf","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9957ea3f43dd6b61f2969ebf: done: 0.017889 sec","job_status":"done","queue_duration_s":1.641907,"scheduling_latency_s":1.641923,"redis_calls":10,"redis_duration_s":0.0035640000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001634,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00193,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017889,"completed_at":"2026-11-10T12:00:12.242Z","load_balancing_strategy":"primary","db_duration_s":0.005876,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 197.925926] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.243Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c5f463407aef1c9595c6e4e9","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c5f463407aef1c9595c6e4e9: start","job_status":"start","queue_duration_s":1.660874,"scheduling_latency_s":1.660889} gitlab # [ 197.967500] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c5f463407aef1c9595c6e4e9","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c5f463407aef1c9595c6e4e9: done: 0.011651 sec","job_status":"done","queue_duration_s":1.660874,"scheduling_latency_s":1.660889,"redis_calls":10,"redis_duration_s":0.0013570000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000816,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00784,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011651,"completed_at":"2026-11-10T12:00:12.255Z","load_balancing_strategy":"primary","db_duration_s":0.001045,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.006276] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"39c9347e265de2331c9cee26","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-39c9347e265de2331c9cee26: start","job_status":"start","queue_duration_s":1.673514,"scheduling_latency_s":1.673528} gitlab # [ 198.007107] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"39c9347e265de2331c9cee26","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-39c9347e265de2331c9cee26: done: 0.011567 sec","job_status":"done","queue_duration_s":1.673514,"scheduling_latency_s":1.673528,"redis_calls":10,"redis_duration_s":0.0012980000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00077,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011567,"completed_at":"2026-11-10T12:00:12.267Z","load_balancing_strategy":"primary","db_duration_s":0.001033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.014604] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b290044a98b9bf801b9cd4c7","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b290044a98b9bf801b9cd4c7: start","job_status":"start","queue_duration_s":1.685974,"scheduling_latency_s":1.685989} gitlab # [ 198.062745] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b290044a98b9bf801b9cd4c7","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b290044a98b9bf801b9cd4c7: done: 0.017372 sec","job_status":"done","queue_duration_s":1.685974,"scheduling_latency_s":1.685989,"redis_calls":10,"redis_duration_s":0.004229,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002358,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001871,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017372,"completed_at":"2026-11-10T12:00:12.285Z","load_balancing_strategy":"primary","db_duration_s":0.003916,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.063725] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3d7acaeaf422cfcb35564c9d","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3d7acaeaf422cfcb35564c9d: start","job_status":"start","queue_duration_s":1.707015,"scheduling_latency_s":1.707032} gitlab # [ 198.104295] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3d7acaeaf422cfcb35564c9d","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3d7acaeaf422cfcb35564c9d: done: 0.012107 sec","job_status":"done","queue_duration_s":1.707015,"scheduling_latency_s":1.707032,"redis_calls":10,"redis_duration_s":0.001374,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000797,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012107,"completed_at":"2026-11-10T12:00:12.301Z","load_balancing_strategy":"primary","db_duration_s":0.001178,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.105475] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"57da7a3b9c731131bd8e79ef","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-57da7a3b9c731131bd8e79ef: start","job_status":"start","queue_duration_s":1.720094,"scheduling_latency_s":1.720108} gitlab # [ 198.150808] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.314Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"57da7a3b9c731131bd8e79ef","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-57da7a3b9c731131bd8e79ef: done: 0.011665 sec","job_status":"done","queue_duration_s":1.720094,"scheduling_latency_s":1.720108,"redis_calls":10,"redis_duration_s":0.001336,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000802,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011665,"completed_at":"2026-11-10T12:00:12.314Z","load_balancing_strategy":"primary","db_duration_s":0.00105,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.152031] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cfaff12a4b5f1f90cef2739e","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cfaff12a4b5f1f90cef2739e: start","job_status":"start","queue_duration_s":1.732624,"scheduling_latency_s":1.732638} gitlab # [ 198.159487] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cfaff12a4b5f1f90cef2739e","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cfaff12a4b5f1f90cef2739e: done: 0.012028 sec","job_status":"done","queue_duration_s":1.732624,"scheduling_latency_s":1.732638,"redis_calls":10,"redis_duration_s":0.001325,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000789,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012028,"completed_at":"2026-11-10T12:00:12.327Z","load_balancing_strategy":"primary","db_duration_s":0.001021,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.223651] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.328Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"39baa36fbc0ba56042593bc0","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-39baa36fbc0ba56042593bc0: start","job_status":"start","queue_duration_s":1.745543,"scheduling_latency_s":1.745558} gitlab # [ 198.224523] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"39baa36fbc0ba56042593bc0","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-39baa36fbc0ba56042593bc0: done: 0.023965 sec","job_status":"done","queue_duration_s":1.745543,"scheduling_latency_s":1.745558,"redis_calls":10,"redis_duration_s":0.006922000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003996,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023965,"completed_at":"2026-11-10T12:00:12.352Z","load_balancing_strategy":"primary","db_duration_s":0.006039,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.238566] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f5682bd60d44166951c6f38e","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f5682bd60d44166951c6f38e: start","job_status":"start","queue_duration_s":1.772935,"scheduling_latency_s":1.77295} gitlab # [ 198.249672] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f5682bd60d44166951c6f38e","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f5682bd60d44166951c6f38e: done: 0.014915 sec","job_status":"done","queue_duration_s":1.772935,"scheduling_latency_s":1.77295,"redis_calls":10,"redis_duration_s":0.003818,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.0023,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014915,"completed_at":"2026-11-10T12:00:12.370Z","load_balancing_strategy":"primary","db_duration_s":0.001115,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.299793] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.371Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3a8b2e1a2f7aca496d98e3a9","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3a8b2e1a2f7aca496d98e3a9: start","job_status":"start","queue_duration_s":1.78874,"scheduling_latency_s":1.788754} gitlab # [ 198.300702] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3a8b2e1a2f7aca496d98e3a9","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3a8b2e1a2f7aca496d98e3a9: done: 0.01149 sec","job_status":"done","queue_duration_s":1.78874,"scheduling_latency_s":1.788754,"redis_calls":10,"redis_duration_s":0.001305,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000757,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01149,"completed_at":"2026-11-10T12:00:12.382Z","load_balancing_strategy":"primary","db_duration_s":0.001003,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.339686] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9cd02cbad5422911425b0602","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9cd02cbad5422911425b0602: start","job_status":"start","queue_duration_s":1.801269,"scheduling_latency_s":1.801283} gitlab # [ 198.340573] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9cd02cbad5422911425b0602","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9cd02cbad5422911425b0602: done: 0.011363 sec","job_status":"done","queue_duration_s":1.801269,"scheduling_latency_s":1.801283,"redis_calls":10,"redis_duration_s":0.001323,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000798,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011363,"completed_at":"2026-11-10T12:00:12.395Z","load_balancing_strategy":"primary","db_duration_s":0.001014,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.393823] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.396Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b57333367e6e04e68483f87b","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b57333367e6e04e68483f87b: start","job_status":"start","queue_duration_s":1.813525,"scheduling_latency_s":1.81354} gitlab # [ 198.394707] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b57333367e6e04e68483f87b","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b57333367e6e04e68483f87b: done: 0.022029 sec","job_status":"done","queue_duration_s":1.813525,"scheduling_latency_s":1.81354,"redis_calls":10,"redis_duration_s":0.006899000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004804,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022029,"completed_at":"2026-11-10T12:00:12.418Z","load_balancing_strategy":"primary","db_duration_s":0.008964,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.418491] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d3e8a7a8a6ffb6735da5bd1a","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d3e8a7a8a6ffb6735da5bd1a: start","job_status":"start","queue_duration_s":1.837548,"scheduling_latency_s":1.837564} gitlab # [ 198.464135] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d3e8a7a8a6ffb6735da5bd1a","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d3e8a7a8a6ffb6735da5bd1a: done: 0.012005 sec","job_status":"done","queue_duration_s":1.837548,"scheduling_latency_s":1.837564,"redis_calls":10,"redis_duration_s":0.001511,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000719,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000792,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012005,"completed_at":"2026-11-10T12:00:12.432Z","load_balancing_strategy":"primary","db_duration_s":0.001183,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.465049] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fde4f2088c2cbd3429be4571","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fde4f2088c2cbd3429be4571: start","job_status":"start","queue_duration_s":1.850433,"scheduling_latency_s":1.850448} gitlab # [ 198.473769] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fde4f2088c2cbd3429be4571","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fde4f2088c2cbd3429be4571: done: 0.011365 sec","job_status":"done","queue_duration_s":1.850433,"scheduling_latency_s":1.850448,"redis_calls":10,"redis_duration_s":0.001323,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000788,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011365,"completed_at":"2026-11-10T12:00:12.444Z","load_balancing_strategy":"primary","db_duration_s":0.001042,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.534860] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3d55be17ee3a0a3a8b211dbd","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3d55be17ee3a0a3a8b211dbd: start","job_status":"start","queue_duration_s":1.862697,"scheduling_latency_s":1.862711} gitlab # [ 198.535759] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.457Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3d55be17ee3a0a3a8b211dbd","created_at":"2026-11-10T12:00:10.536Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-1154edde90cd4d46","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"434fb7cd629579234937d969e149f11a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:10.582Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3d55be17ee3a0a3a8b211dbd: done: 0.0119 sec","job_status":"done","queue_duration_s":1.862697,"scheduling_latency_s":1.862711,"redis_calls":10,"redis_duration_s":0.001305,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000779,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0119,"completed_at":"2026-11-10T12:00:12.457Z","load_balancing_strategy":"primary","db_duration_s":0.001023,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.577550] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.461Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"7d1b956c197d4b19d51e0135","created_at":"2026-11-10T12:00:11.201Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-b66e1647c5534ee7","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"c5208bda83e467491d0e51b9daed7ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:11.210Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-7d1b956c197d4b19d51e0135: start","job_status":"start","queue_duration_s":1.25087,"scheduling_latency_s":1.250885} gitlab # [ 198.578503] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.483Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"7d1b956c197d4b19d51e0135","created_at":"2026-11-10T12:00:11.201Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-b66e1647c5534ee7","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"c5208bda83e467491d0e51b9daed7ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:11.210Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-7d1b956c197d4b19d51e0135: done: 0.021743 sec","job_status":"done","queue_duration_s":1.25087,"scheduling_latency_s":1.250885,"redis_calls":18,"redis_duration_s":0.004281,"redis_read_bytes":39,"redis_write_bytes":2116,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000394,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002176,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":471,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.001711,"redis_shared_state_read_bytes":34,"redis_shared_state_write_bytes":1324,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000215111,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":0,"duration_s":0.021743,"completed_at":"2026-11-10T12:00:12.483Z","load_balancing_strategy":"primary","db_duration_s":0.002203,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.622685] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"77475b30eb7d2f844b7df284","created_at":"2026-11-10T12:00:11.201Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-b66e1647c5534ee7","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"c5208bda83e467491d0e51b9daed7ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:11.210Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-77475b30eb7d2f844b7df284: start","job_status":"start","queue_duration_s":1.273562,"scheduling_latency_s":1.273575} gitlab # [ 198.623668] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"77475b30eb7d2f844b7df284","created_at":"2026-11-10T12:00:11.201Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-b66e1647c5534ee7","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"c5208bda83e467491d0e51b9daed7ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:11.210Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-77475b30eb7d2f844b7df284: done: 0.013782 sec","job_status":"done","queue_duration_s":1.273562,"scheduling_latency_s":1.273575,"redis_calls":15,"redis_duration_s":0.001767,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.001192,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":2,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":2,"duration_s":0.013782,"completed_at":"2026-11-10T12:00:12.497Z","load_balancing_strategy":"primary","db_duration_s":0.001303,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.651633] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.499Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"29805c57f7c6d4fbabd66099","created_at":"2026-11-10T12:00:11.201Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-b66e1647c5534ee7","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"c5208bda83e467491d0e51b9daed7ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:11.210Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-29805c57f7c6d4fbabd66099: start","job_status":"start","queue_duration_s":1.288639,"scheduling_latency_s":1.288654} gitlab # [ 198.663145] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.516Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"29805c57f7c6d4fbabd66099","created_at":"2026-11-10T12:00:11.201Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-b66e1647c5534ee7","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"c5208bda83e467491d0e51b9daed7ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:11.210Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-29805c57f7c6d4fbabd66099: done: 0.016954 sec","job_status":"done","queue_duration_s":1.288639,"scheduling_latency_s":1.288654,"redis_calls":15,"redis_duration_s":0.004786,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.003267,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009201,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":4,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":4,"duration_s":0.016954,"completed_at":"2026-11-10T12:00:12.516Z","load_balancing_strategy":"primary","db_duration_s":0.001465,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.721832] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.517Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"fe1998d70cae5edb96436761","created_at":"2026-11-10T12:00:11.201Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-b66e1647c5534ee7","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"c5208bda83e467491d0e51b9daed7ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:11.210Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-fe1998d70cae5edb96436761: start","job_status":"start","queue_duration_s":1.306762,"scheduling_latency_s":1.306776} gitlab # [ 198.722625] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.531Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"fe1998d70cae5edb96436761","created_at":"2026-11-10T12:00:11.201Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-b66e1647c5534ee7","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"c5208bda83e467491d0e51b9daed7ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:11.210Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-fe1998d70cae5edb96436761: done: 0.013971 sec","job_status":"done","queue_duration_s":1.306762,"scheduling_latency_s":1.306776,"redis_calls":15,"redis_duration_s":0.002011,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000759,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.001252,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":2,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":2,"duration_s":0.013971,"completed_at":"2026-11-10T12:00:12.531Z","load_balancing_strategy":"primary","db_duration_s":0.001413,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.751523] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"7bcda84985a143f7a7168f9a","created_at":"2026-11-10T12:00:11.201Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-b66e1647c5534ee7","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"c5208bda83e467491d0e51b9daed7ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:11.210Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-7bcda84985a143f7a7168f9a: start","job_status":"start","queue_duration_s":1.321621,"scheduling_latency_s":1.321636} gitlab # [ 198.793714] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:12.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::BulkDeleteExpiredJobArtifactsWorker","args":[],"jid":"7bcda84985a143f7a7168f9a","created_at":"2026-11-10T12:00:11.201Z","trace_propagation_headers":{"sentry-trace":"daf8a73441c34a2f8e8db157baab27c5-b66e1647c5534ee7","baggage":"sentry-trace_id=daf8a73441c34a2f8e8db157baab27c5,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleBulkDeleteJobArtifactCronWorker","correlation_id":"c5208bda83e467491d0e51b9daed7ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:11.210Z","job_size_bytes":2,"pid":1650,"sidekiq_tid":"maiy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::BulkDeleteExpiredJobArtifactsWorker JID-7bcda84985a143f7a7168f9a: done: 0.013354 sec","job_status":"done","queue_duration_s":1.321621,"scheduling_latency_s":1.321636,"redis_calls":15,"redis_duration_s":0.0017040000000000002,"redis_read_bytes":35,"redis_write_bytes":1707,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":383,"redis_shared_state_calls":11,"redis_shared_state_duration_s":0.001167,"redis_shared_state_read_bytes":32,"redis_shared_state_write_bytes":1324,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008896,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket":3,"extra.ci_bulk_delete_expired_job_artifacts_worker.artifacts_empty":true,"extra.ci_bulk_delete_expired_job_artifacts_worker.destroyed_job_artifacts_count":0,"extra.ci_bulk_delete_expired_job_artifacts_worker.mod_bucket_released":3,"duration_s":0.013354,"completed_at":"2026-11-10T12:00:12.545Z","load_balancing_strategy":"primary","db_duration_s":0.001248,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 198.801974] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:16.879Z","message":"Shutting down"} gitlab # [ 198.802992] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:16.881Z","message":"Scheduler exiting..."} gitlab # [ 198.804421] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:16.881Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 198.805760] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:16.881Z","message":"Scheduler exiting..."} gitlab # [ 198.807376] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 198.808117] setup-gitlab-start[2283]: + systemctl restart gitlab-sidekiq.service gitlab # [ 198.808767] runuser[2305]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 199.002274] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 6574970737 wd_nsec: 6574967595 gitlab # [ 199.251159] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:17.382Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 199.753779] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:17.884Z","message":"Bye!"} gitlab # [ 199.778747] sidekiq-cluster[1650]: {"severity":"INFO","time":"2026-11-10T12:00:17.910Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1650,"worker_id":"sidekiq_0","memwd_rss_bytes":950685696,"retry":0} gitlab # [ 200.320129] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 200.322750] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 200.324477] systemd[1]: gitlab-sidekiq.service: Consumed 1min 5.891s CPU time over 2min 3.518s wall clock time, 1005M memory peak, 13.1M read from disk. gitlab # [ 200.334240] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 200.339387] setup-gitlab-start[2283]: + mkdir -p /var/lib/secrets gitlab # [ 200.345190] setup-gitlab-start[2283]: + echo nixbot-test-token-of-20-chars gitlab # [ 200.347652] setup-gitlab-start[2283]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 200.350204] setup-gitlab-start[2283]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars gitlab # [ 200.735939] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:18 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 356 gitlab # [ 201.286176] gitaly[919]: time="2026-11-10T12:00:19.416Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T12:00:29.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.416" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.286875] gitaly[919]: time="2026-11-10T12:00:19.417Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T12:00:29.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.416" grpc.time_ms=0.464 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.430383] gitaly[919]: time="2026-11-10T12:00:19.562Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T12:00:29.151" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.562" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.430853] gitaly[919]: time="2026-11-10T12:00:19.562Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T12:00:29.151" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.562" grpc.time_ms=0.438 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.540896] gitaly[919]: time="2026-11-10T12:00:19.672Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD diskcache=968c107d-a261-48f6-b843-13c996792ed7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T12:00:29.204" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.614" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.559469] gitaly[919]: time="2026-11-10T12:00:19.691Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1576 command.oublock=32 command.real_time_ms=26 command.spawn_token_fork_ms=1 command.system_time_ms=16 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T12:00:29.204" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.614" grpc.time_ms=58.309 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.591092] gitaly[919]: time="2026-11-10T12:00:19.723Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T12:00:29.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.722" pid=919 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 201.591828] gitaly[919]: time="2026-11-10T12:00:19.723Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T12:00:29.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.722" grpc.time_ms=0.462 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.627096] gitaly[919]: time="2026-11-10T12:00:19.759Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=618 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T12:00:29.026" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.747" grpc.time_ms=11.972 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.690767] gitaly[919]: time="2026-11-10T12:00:19.822Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T12:00:29.101" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.822" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.691742] gitaly[919]: time="2026-11-10T12:00:19.823Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T12:00:29.101" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.822" grpc.time_ms=0.392 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.779108] gitaly[919]: time="2026-11-10T12:00:19.911Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD diskcache=3af1c79c-dd5f-4c5b-ad7b-f82857ee5591 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T12:00:29.143" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.863" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.779883] gitaly[919]: time="2026-11-10T12:00:19.911Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1577 command.oublock=32 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T12:00:29.143" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.863" grpc.time_ms=47.347 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.814650] gitaly[919]: time="2026-11-10T12:00:19.946Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T12:00:29.225" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.946" pid=919 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 201.815527] gitaly[919]: time="2026-11-10T12:00:19.946Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T12:00:29.225" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.946" grpc.time_ms=0.471 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.848075] gitaly[919]: time="2026-11-10T12:00:19.980Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T12:00:29.249" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:19.970" grpc.time_ms=9.24 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.899042] gitaly[919]: time="2026-11-10T12:00:20.031Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T12:00:30.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:20.030" grpc.time_ms=0.144 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.911391] gitaly[919]: time="2026-11-10T12:00:20.043Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T12:00:30.022" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:20.043" grpc.time_ms=0.384 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 202.408965] gitaly[919]: time="2026-11-10T12:00:20.540Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72N96Y78W46XNW7TQ89SSRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T12:00:30.123" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:20.534" grpc.time_ms=5.783 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 202.718312] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:20 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 1961 gitlab # [ 202.722834] setup-gitlab-start[2573]: {"id":1,"message":null,"error":null} gitlab # [ 203.569284] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-11-10T12:00:21.682Z","message":"Starting cluster with 1 processes"} gitlab # [ 208.097239] gitaly[919]: time="2026-11-10T12:00:26.228Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9DRWZ9PZEY1WPH8Z3G75 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T12:00:36.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:26.217" grpc.time_ms=10.634 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 208.533097] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:26 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 779 gitlab # [ 208.537838] setup-gitlab-start[2283]: + mkdir -p /tmp/test-flake gitlab # [ 208.541252] setup-gitlab-start[2283]: + cd /tmp/test-flake gitlab # [ 208.541977] setup-gitlab-start[2283]: + git init -b master gitlab # [ 208.564094] setup-gitlab-start[2655]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 208.564797] setup-gitlab-start[2283]: + git config user.name test gitlab # [ 208.566545] setup-gitlab-start[2283]: + git config user.email test@example.com gitlab # [ 208.568739] setup-gitlab-start[2283]: + cat gitlab # [ 208.570989] setup-gitlab-start[2283]: + git add flake.nix gitlab # [ 208.573667] setup-gitlab-start[2283]: + git commit -m 'initial commit' gitlab # [ 208.594521] setup-gitlab-start[2660]: [master (root-commit) bdc26eb] initial commit gitlab # [ 208.594903] setup-gitlab-start[2660]: 1 file changed, 10 insertions(+) gitlab # [ 208.595469] setup-gitlab-start[2660]: create mode 100644 flake.nix gitlab # [ 208.596328] setup-gitlab-start[2283]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 208.604308] setup-gitlab-start[2283]: + git push -u origin master gitlab # [ 208.658731] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:26 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 30 gitlab # [ 208.753564] gitaly[919]: time="2026-11-10T12:00:26.885Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72N9EN8RZF8K33DC3RDANQ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:26.874" grpc.time_ms=10.654 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 208.754733] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:26 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 92 gitlab # [ 208.969510] gitaly[919]: time="2026-11-10T12:00:27.101Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T12:00:37.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:27.090" grpc.time_ms=10.766 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.028764] gitaly[919]: time="2026-11-10T12:00:27.160Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-10T12:00:57.054" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-10T12:00:27.154" grpc.time_ms=6.405 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.077742] gitaly[919]: time="2026-11-10T12:00:27.209Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB duration_ms=179ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 209.111528] gitaly[919]: time="2026-11-10T12:00:27.242Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB duration_ms=27ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 209.112290] gitaly[919]: time="2026-11-10T12:00:27.243Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:00:27.029" grpc.time_ms=214.103 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.182467] gitaly[919]: time="2026-11-10T12:00:27.314Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:00:27.313" grpc.time_ms=0.955 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.248441] gitaly[919]: time="2026-11-10T12:00:27.380Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:00:27.379" grpc.time_ms=1.058 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.306383] gitaly[919]: time="2026-11-10T12:00:27.438Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:00:27.437" grpc.time_ms=0.522 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.409611] gitaly[919]: time="2026-11-10T12:00:27.541Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T12:00:37.120" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:27.531" grpc.time_ms=10.434 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.464538] gitaly[919]: time="2026-11-10T12:00:27.596Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1567 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-10T12:00:37.171" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:00:27.582" grpc.time_ms=14.156 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 209.498909] gitaly[919]: time="2026-11-10T12:00:27.630Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB duration_ms=146ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 209.502101] gitaly[919]: time="2026-11-10T12:00:27.634Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:00:27.483" grpc.time_ms=150.279 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.524532] gitaly[919]: time="2026-11-10T12:00:27.656Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB diskcache=b94ec58f-b73e-4d44-8d0f-b9070f5d8b79 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:26.975" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.525319] gitaly[919]: time="2026-11-10T12:00:27.656Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=218 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13800 command.oublock=32 command.real_time_ms=668 command.spawn_token_fork_ms=0 command.system_time_ms=136 command.user_time_ms=82 component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:26.975" grpc.time_ms=680.924 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 209.547129] gitlab-workhorse[825]: time="2026-11-10T12:00:27Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 209.547844] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:27 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 777 gitlab # [ 209.552672] setup-gitlab-start[2664]: To http://gitlab/root/test-flake.git gitlab # [ 209.553241] setup-gitlab-start[2664]: * [new branch] master -> master gitlab # [ 209.559310] setup-gitlab-start[2664]: branch 'master' set up to track 'origin/master'. gitlab # [ 209.563391] systemd[1]: Finished setup-gitlab.service. gitlab # [ 209.567376] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 210.61 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 211.104495] nixbot[2744]: {"ts": "2026-11-10T12:00:29.234Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 211.169905] nixbot[2744]: {"ts": "2026-11-10T12:00:29.301Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 211.208239] nixbot[2744]: {"ts": "2026-11-10T12:00:29.339Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 211.219821] nixbot[2744]: {"ts": "2026-11-10T12:00:29.351Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 211.235814] nixbot[2744]: {"ts": "2026-11-10T12:00:29.367Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 211.256658] nixbot[2744]: {"ts": "2026-11-10T12:00:29.388Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 211.270096] nixbot[2744]: {"ts": "2026-11-10T12:00:29.401Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 211.281435] nixbot[2744]: {"ts": "2026-11-10T12:00:29.413Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 211.305736] nixbot[2744]: {"ts": "2026-11-10T12:00:29.437Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 211.315630] nixbot[2744]: {"ts": "2026-11-10T12:00:29.447Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 211.334227] nixbot[2744]: {"ts": "2026-11-10T12:00:29.466Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 211.360125] nixbot[2744]: {"ts": "2026-11-10T12:00:29.491Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 211.370104] nixbot[2744]: {"ts": "2026-11-10T12:00:29.501Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 211.381112] nixbot[2744]: {"ts": "2026-11-10T12:00:29.512Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 211.394618] nixbot[2744]: {"ts": "2026-11-10T12:00:29.526Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 211.405424] nixbot[2744]: {"ts": "2026-11-10T12:00:29.537Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 211.417460] nixbot[2744]: {"ts": "2026-11-10T12:00:29.549Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 211.427316] nixbot[2744]: {"ts": "2026-11-10T12:00:29.559Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 211.442960] nixbot[2744]: {"ts": "2026-11-10T12:00:29.574Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 211.451911] nixbot[2744]: {"ts": "2026-11-10T12:00:29.583Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 211.462430] nixbot[2744]: {"ts": "2026-11-10T12:00:29.594Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 211.478699] nixbot[2744]: {"ts": "2026-11-10T12:00:29.610Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 211.486636] nixbot[2744]: {"ts": "2026-11-10T12:00:29.618Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 211.495330] nixbot[2744]: {"ts": "2026-11-10T12:00:29.627Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 211.504762] nixbot[2744]: {"ts": "2026-11-10T12:00:29.636Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 211.515239] nixbot[2744]: {"ts": "2026-11-10T12:00:29.647Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 211.526566] nixbot[2744]: {"ts": "2026-11-10T12:00:29.658Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 211.536825] nixbot[2744]: {"ts": "2026-11-10T12:00:29.668Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 211.546168] nixbot[2744]: {"ts": "2026-11-10T12:00:29.678Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 211.578191] nixbot[2744]: {"ts": "2026-11-10T12:00:29.709Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 211.594732] nixbot[2744]: {"ts": "2026-11-10T12:00:29.726Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 211.605690] nixbot[2744]: {"ts": "2026-11-10T12:00:29.737Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 211.616309] nixbot[2744]: {"ts": "2026-11-10T12:00:29.747Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} gitlab # [ 211.630301] nixbot[2744]: {"ts": "2026-11-10T12:00:29.762Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"} gitlab # [ 211.640341] nixbot[2744]: {"ts": "2026-11-10T12:00:29.772Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"} gitlab # [ 211.662836] nixbot[2744]: {"ts": "2026-11-10T12:00:29.794Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"} gitlab # [ 211.675951] nixbot[2744]: {"ts": "2026-11-10T12:00:29.807Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"} gitlab # [ 211.692824] nixbot[2744]: {"ts": "2026-11-10T12:00:29.824Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "effect_when_changed"} gitlab # [ 211.706746] nixbot[2744]: {"ts": "2026-11-10T12:00:29.838Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 38, "migration": "commit_statuses"} gitlab # [ 212.140957] nixbot[2744]: {"ts": "2026-11-10T12:00:30.272Z", "level": "info", "logger": "uvicorn.error", "msg": "Started server process [2744]", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"} gitlab # [ 212.141885] nixbot[2744]: {"ts": "2026-11-10T12:00:30.272Z", "level": "info", "logger": "uvicorn.error", "msg": "Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)"} gitlab # [ 212.164527] nixbot[2744]: {"ts": "2026-11-10T12:00:30.296Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:55716 - \"GET /health HTTP/1.1\" 200"} gitlab # [ 212.171264] systemd[1]: Started nixbot CI service. gitlab # [ 212.173752] systemd[1]: Reached target Multi-User System. gitlab # [ 212.175223] systemd[1]: Startup finished in 1.215s (kernel) + 3.687s (initrd) + 3min 27.272s (userspace) = 3min 32.174s. gitlab # [ 212.373473] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:30 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 208 gitlab # [ 212.395660] nixbot[2744]: {"ts": "2026-11-10T12:00:30.527Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 212.479307] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 69 gitlab # [ 212.481563] nixbot[2744]: {"ts": "2026-11-10T12:00:30.613Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 212.602932] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 119 gitlab # [ 212.717911] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:30 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 102 gitlab # [ 212.766512] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 39 gitlab # [ 212.883505] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:31 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 105 gitlab # [ 213.045750] gitaly[919]: time="2026-11-10T12:00:31.177Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9JSH6XY6XNA9TN71FNBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-11-10T12:01:01.059" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:00:31.160" grpc.time_ms=16.296 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.079557] gitaly[919]: time="2026-11-10T12:00:31.198Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9JSH6XY6XNA9TN71FNBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:01.096" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:00:31.197" grpc.time_ms=1.516 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: waiting for unit nixbot.service, in 3.13 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 213.111680] gitaly[919]: time="2026-11-10T12:00:31.243Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9JSH6XY6XNA9TN71FNBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-11-10T12:00:41.195" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:00:31.234" grpc.time_ms=8.74 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.133496] nixbot[2744]: {"ts": "2026-11-10T12:00:31.265Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47296 - \"GET /health HTTP/1.1\" 200"} gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.02 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 213.149802] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 256 gitlab # [ 213.199035] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 45 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.07 seconds) (finished: subtest: project discovered and webhook registered, in 0.07 seconds) subtest: push delivers the webhook gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' gitlab # [ 213.218948] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 65 gitlab # [ 213.224569] nixbot[2744]: {"ts": "2026-11-10T12:00:31.356Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 213.330961] gitaly[919]: time="2026-11-10T12:00:31.462Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72N9K5JC0YWB0YH3BVYYBBY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:31.452" grpc.time_ms=9.986 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.361738] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 83 gitlab # [ 213.447592] gitaly[919]: time="2026-11-10T12:00:31.579Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72N9K8B34YN0RZXQY3R30BA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:31.566" pid=919 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 213.469238] gitaly[919]: time="2026-11-10T12:00:31.579Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9K8B34YN0RZXQY3R30BA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:31.566" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.495149] gitaly[919]: time="2026-11-10T12:00:31.627Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9K8B34YN0RZXQY3R30BA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:31.566" grpc.time_ms=59.995 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.496391] gitlab-workhorse[825]: time="2026-11-10T12:00:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 213.496837] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:31 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 128 gitlab # [ 213.629922] gitaly[919]: time="2026-11-10T12:00:31.761Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KCH5JRN3EM4B2V0RQ8T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:31.737" pid=919 remote_ip="2001:db8:1::1" request_sha=b2910d2201b5c8b544d8ed3772c677035a77821e response_bytes=223 span.kind=server system=grpc gitlab # [ 213.652873] gitaly[919]: time="2026-11-10T12:00:31.761Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1206 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KCH5JRN3EM4B2V0RQ8T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:31.737" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.675305] gitlab-workhorse[825]: time="2026-11-10T12:00:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=223 gitlab # [ 213.677146] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:31 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 128 gitlab # [ 213.679141] gitaly[919]: time="2026-11-10T12:00:31.761Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1206 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KCH5JRN3EM4B2V0RQ8T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:31.737" grpc.time_ms=23.892 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.738778] gitaly[919]: time="2026-11-10T12:00:31.870Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72N9KJQ8Q9WE0EQ4332AQRX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:31.864" grpc.time_ms=6.793 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.750754] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 40 gitlab # [ 213.785772] gitaly[919]: time="2026-11-10T12:00:31.917Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KM1RM3V257R0Y5AMVGT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:31.906" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 213.794929] gitaly[919]: time="2026-11-10T12:00:31.917Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KM1RM3V257R0Y5AMVGT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:31.906" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.806972] gitaly[919]: time="2026-11-10T12:00:31.939Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KM1RM3V257R0Y5AMVGT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:31.906" grpc.time_ms=32.107 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.820416] gitlab-workhorse[825]: time="2026-11-10T12:00:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 213.820755] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:31 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 78 gitlab # [ 213.887394] gitaly[919]: time="2026-11-10T12:00:32.019Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72N9KQD1C7PR56Z25X1PQJ1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:32.012" grpc.time_ms=6.536 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.903882] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:32 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 54 gitlab # [ 213.952289] gitaly[919]: time="2026-11-10T12:00:32.084Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KS56QPZ5TX1FFC0JYNC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:32.072" pid=919 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 213.971286] gitaly[919]: time="2026-11-10T12:00:32.084Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KS56QPZ5TX1FFC0JYNC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:32.072" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 213.977536] gitaly[919]: time="2026-11-10T12:00:32.109Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1175 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KS56QPZ5TX1FFC0JYNC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:32.072" grpc.time_ms=36.722 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 214.000602] gitlab-workhorse[825]: time="2026-11-10T12:00:32Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 214.001619] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:32 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 94 gitlab # [ 214.097384] gitaly[919]: time="2026-11-10T12:00:32.228Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:02.127" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:00:32.227" grpc.time_ms=0.68 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.158560] gitaly[919]: time="2026-11-10T12:00:32.290Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-10T12:01:02.179" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:00:32.280" grpc.time_ms=10.082 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.214935] gitaly[919]: time="2026-11-10T12:00:32.347Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:00:42.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:00:32.335" grpc.time_ms=11.631 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.264088] gitaly[919]: time="2026-11-10T12:00:32.395Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:00:42.052" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:00:32.383" grpc.time_ms=12.046 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.318538] gitaly[919]: time="2026-11-10T12:00:32.450Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T12:00:42.039" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:32.450" grpc.time_ms=0.435 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.355162] gitaly[919]: time="2026-11-10T12:00:32.487Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T12:00:42.065" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:00:32.476" grpc.time_ms=10.836 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.416810] gitaly[919]: time="2026-11-10T12:00:32.548Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:00:32.548" grpc.time_ms=0.537 pid=919 span.kind=server system=grpc gitlab # [ 214.453732] gitaly[919]: time="2026-11-10T12:00:32.585Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:00:32.585" grpc.time_ms=0.723 pid=919 span.kind=server system=grpc gitlab # [ 214.472229] gitaly[919]: time="2026-11-10T12:00:32.604Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 diskcache=df1af06a-ea83-4eed-9098-367355bcdf6c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T12:00:42.095" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:32.506" pid=919 ref=refs/keep-around/bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 remote_ip="2001:db8:1::1" revision=bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 span.kind=server system=grpc username=root gitlab # [ 214.482677] gitaly[919]: time="2026-11-10T12:00:32.614Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=79 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5402 command.oublock=8 command.real_time_ms=88 command.spawn_token_fork_ms=0 command.system_time_ms=53 command.user_time_ms=26 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T12:00:42.095" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:00:32.506" grpc.time_ms=98.431 pid=919 ref=refs/keep-around/bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 remote_ip="2001:db8:1::1" revision=bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 span.kind=server system=grpc username=root gitlab # [ 214.486898] gitaly[919]: time="2026-11-10T12:00:32.619Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:02.517" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:00:32.618" grpc.time_ms=0.459 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.622134] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:32 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 573 gitlab # [ 214.644344] nixbot[2744]: {"ts": "2026-11-10T12:00:32.775Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-bdc26eb0a0f4-0"} gitlab # [ 214.867757] nixbot[2744]: {"ts": "2026-11-10T12:00:32.999Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-bdc26eb0a0f4-0", "argv0": "bwrap"} gitlab # [ 214.968584] systemd[1]: Started Nix Daemon. gitlab # [ 215.041861] nix-daemon[2905]: accepted connection from pid 2900, user nixbot gitlab # [ 219.255165] nix-daemon[2905]: accepted connection from pid 2947, user nixbot gitlab # [ 219.273378] nix-daemon[2905]: accepted connection from pid 2900, user nixbot gitlab # [ 223.659759] nix-daemon[2905]: accepted connection from pid 2900, user nixbot gitlab # [ 223.758215] nix-daemon[2905]: accepted connection from pid 2996, user nixbot gitlab # [ 223.812411] gitaly[919]: time="2026-11-10T12:00:41.943Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9XCJVE803GNBKMGR4VVD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:11.841" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:00:41.942" grpc.time_ms=0.846 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 223.870729] gitaly[919]: time="2026-11-10T12:00:42.002Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72N9XCJVE803GNBKMGR4VVD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:00:51.271" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:00:41.991" grpc.time_ms=10.923 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 223.917521] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:42 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 174 gitlab # [ 223.973592] gitaly[919]: time="2026-11-10T12:00:42.105Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9XJDY2GVDKKX0127KY8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:12.003" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:00:42.104" grpc.time_ms=0.827 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 224.152159] gitaly[919]: time="2026-11-10T12:00:42.284Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72N9XJDY2GVDKKX0127KY8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:00:52.062" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:00:42.273" grpc.time_ms=10.927 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 224.302878] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:42 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 372 gitlab # [ 228.744751] nix-daemon[2905]: accepted connection from pid 3051, user nixbot gitlab # [ 228.827854] gitaly[919]: time="2026-11-10T12:00:46.959Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NA29PT27AX5DJTS8N43DS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:16.858" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:00:46.958" grpc.time_ms=0.605 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 228.888982] gitaly[919]: time="2026-11-10T12:00:47.021Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NA29PT27AX5DJTS8N43DS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:00:57.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:00:47.010" grpc.time_ms=10.767 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 228.947385] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:47 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 175 gitlab # [ 229.181977] nixbot[2744]: {"ts": "2026-11-10T12:00:47.313Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 1, "sha": "bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []} gitlab # [ 229.305525] gitaly[919]: time="2026-11-10T12:00:47.437Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NA2R1SGTFCEZVXXRQNNDX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:47.427" grpc.time_ms=9.722 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 229.336136] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 106 gitlab # [ 229.416487] gitaly[919]: time="2026-11-10T12:00:47.548Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72NA2VJNGS07JEAT1G7FRX4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:47.535" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 229.416953] gitaly[919]: time="2026-11-10T12:00:47.548Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72NA2VJNGS07JEAT1G7FRX4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:47.535" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 229.418478] gitaly[919]: time="2026-11-10T12:00:47.548Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72NA2VJNGS07JEAT1G7FRX4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:47.535" grpc.time_ms=13.025 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 229.444915] gitlab-workhorse[825]: time="2026-11-10T12:00:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 229.446347] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 102 gitlab # [ 229.538064] gitaly[919]: time="2026-11-10T12:00:47.670Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NA30D3ZEM10QMXHT4TGPZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:47.663" grpc.time_ms=6.755 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 229.538803] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 41 gitlab # [ 229.584132] gitaly[919]: time="2026-11-10T12:00:47.716Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72NA31R05PZA0MWP6PV2YAB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:47.709" pid=919 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 229.584357] gitaly[919]: time="2026-11-10T12:00:47.716Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NA31R05PZA0MWP6PV2YAB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:47.709" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 229.605638] gitaly[919]: time="2026-11-10T12:00:47.737Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NA31R05PZA0MWP6PV2YAB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:00:47.709" grpc.time_ms=27.737 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 229.606161] gitlab-workhorse[825]: time="2026-11-10T12:00:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 229.606711] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:00:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 65 gitlab # [ 232.977589] sidekiq-cluster[2616]: 2026-11-10T12:00:51.109Z pid=2616 tid=xao INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2616, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>434442240} gitlab # [ 233.163691] sidekiq-cluster[2616]: W, [2026-11-10T12:00:51.295632 #2616] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 233.202770] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:00:51.334Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 235.157655] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:00:53.288Z","message":"GitLab reliable fetch activated!"} gitlab # [ 235.158240] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:00:53.289Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 257.075350] postgres[3328]: [3328] ERROR: relation "table_without_models" does not exist gitlab # [ 257.076185] postgres[3328]: [3328] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 259.724174] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:17.855Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 259.725073] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:17.857Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 259.726692] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:17.857Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 259.728306] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:17.857Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 259.729448] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:17.858Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 46.77 seconds) gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 260.080652] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:18 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 42 gitlab # [ 260.160258] gitaly[919]: time="2026-11-10T12:01:18.292Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB0WBSCNM8NDPNW4HYFK0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:18.281" grpc.time_ms=10.236 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 260.162206] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:18 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 74 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 260.433491] gitaly[919]: time="2026-11-10T12:01:18.565Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-10T12:01:48.453" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-10T12:01:18.553" grpc.time_ms=11.518 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 260.480357] gitaly[919]: time="2026-11-10T12:01:18.612Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M72NB10ANG9CDV34HT585920 duration_ms=101ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 260.529099] gitaly[919]: time="2026-11-10T12:01:18.661Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M72NB10ANG9CDV34HT585920 duration_ms=43ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 260.535584] gitaly[919]: time="2026-11-10T12:01:18.667Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:18.509" grpc.time_ms=158.017 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 260.607986] gitaly[919]: time="2026-11-10T12:01:18.740Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:18.737" grpc.time_ms=2.201 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 260.678360] gitaly[919]: time="2026-11-10T12:01:18.809Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:18.807" grpc.time_ms=2.598 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 260.725861] gitaly[919]: time="2026-11-10T12:01:18.857Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:18.857" grpc.time_ms=0.592 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 260.836619] gitaly[919]: time="2026-11-10T12:01:18.968Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M72NB10ANG9CDV34HT585920 duration_ms=65ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 260.843401] gitaly[919]: time="2026-11-10T12:01:18.975Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:18.902" grpc.time_ms=66.591 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 260.881213] gitaly[919]: time="2026-11-10T12:01:19.013Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 diskcache=4c615fd8-b300-4e7d-b089-f71008914793 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:18.443" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 260.898825] gitaly[919]: time="2026-11-10T12:01:19.013Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=288 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13721 command.oublock=32 command.real_time_ms=559 command.spawn_token_fork_ms=1 command.system_time_ms=179 command.user_time_ms=109 component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:18.443" grpc.time_ms=570.32 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 260.899314] gitlab-workhorse[825]: time="2026-11-10T12:01:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 260.899552] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:19 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 666 gitlab # To http://gitlab/root/test-flake.git gitlab # bdc26eb..6279a85 master -> master gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.94 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa' gitlab # [ 261.000964] nixbot[2744]: {"ts": "2026-11-10T12:01:19.132Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47154 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa HTTP/1.1\" 200"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa', in 0.08 seconds) gitlab # [ 261.419686] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:19.550Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c6802c4037dfe41518350238","created_at":"2026-11-10T12:00:19.773Z","trace_propagation_headers":{"sentry-trace":"6fb82f98c63f496fb522c81f4255b94e-a19c71b0ca304e9f","baggage":"sentry-trace_id=6fb82f98c63f496fb522c81f4255b94e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N96Y78W46XNW7TQ89SSRD","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:19.805Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c6802c4037dfe41518350238: start","job_status":"start","queue_duration_s":59.744652,"scheduling_latency_s":59.744736} gitlab # [ 261.637600] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:19.769Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c6802c4037dfe41518350238","created_at":"2026-11-10T12:00:19.773Z","trace_propagation_headers":{"sentry-trace":"6fb82f98c63f496fb522c81f4255b94e-a19c71b0ca304e9f","baggage":"sentry-trace_id=6fb82f98c63f496fb522c81f4255b94e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N96Y78W46XNW7TQ89SSRD","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:19.805Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c6802c4037dfe41518350238: done: 0.219081 sec","job_status":"done","queue_duration_s":59.744652,"scheduling_latency_s":59.744736,"redis_calls":25,"redis_duration_s":0.008024,"redis_read_bytes":912,"redis_write_bytes":2805,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.004977,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001709,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001338,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":11,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0828745,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.219081,"completed_at":"2026-11-10T12:01:19.769Z","load_balancing_strategy":"primary","db_duration_s":0.049241,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 261.715718] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:19.846Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"7c1fad9d4acdf267e6730fc0","created_at":"2026-11-10T12:00:19.814Z","trace_propagation_headers":{"sentry-trace":"6fb82f98c63f496fb522c81f4255b94e-a19c71b0ca304e9f","baggage":"sentry-trace_id=6fb82f98c63f496fb522c81f4255b94e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N96Y78W46XNW7TQ89SSRD","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:19.818Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-7c1fad9d4acdf267e6730fc0: start","job_status":"start","queue_duration_s":60.028405,"scheduling_latency_s":60.028465} gitlab # [ 261.848862] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:19.980Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"7c1fad9d4acdf267e6730fc0","created_at":"2026-11-10T12:00:19.814Z","trace_propagation_headers":{"sentry-trace":"6fb82f98c63f496fb522c81f4255b94e-a19c71b0ca304e9f","baggage":"sentry-trace_id=6fb82f98c63f496fb522c81f4255b94e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N96Y78W46XNW7TQ89SSRD","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:19.818Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-7c1fad9d4acdf267e6730fc0: done: 0.133856 sec","job_status":"done","queue_duration_s":60.028405,"scheduling_latency_s":60.028465,"redis_calls":14,"redis_duration_s":0.003724,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000827,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001607,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00129,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.085617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.091008774,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.133856,"completed_at":"2026-11-10T12:01:19.980Z","load_balancing_strategy":"primary","db_duration_s":0.026231,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 261.925774] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:20.057Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ad1d1ac850840d231592f6dc","created_at":"2026-11-10T12:00:20.453Z","trace_propagation_headers":{"sentry-trace":"6fb82f98c63f496fb522c81f4255b94e-a19c71b0ca304e9f","baggage":"sentry-trace_id=6fb82f98c63f496fb522c81f4255b94e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N96Y78W46XNW7TQ89SSRD","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:20.458Z","job_size_bytes":5,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ad1d1ac850840d231592f6dc: start","job_status":"start","queue_duration_s":59.599288,"scheduling_latency_s":59.599331} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa' gitlab # [ 262.045317] nixbot[2744]: {"ts": "2026-11-10T12:01:20.176Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47156 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa HTTP/1.1\" 200"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa', in 0.04 seconds) gitlab # [ 262.096297] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:20.227Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ad1d1ac850840d231592f6dc","created_at":"2026-11-10T12:00:20.453Z","trace_propagation_headers":{"sentry-trace":"6fb82f98c63f496fb522c81f4255b94e-a19c71b0ca304e9f","baggage":"sentry-trace_id=6fb82f98c63f496fb522c81f4255b94e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N96Y78W46XNW7TQ89SSRD","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:20.458Z","job_size_bytes":5,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ad1d1ac850840d231592f6dc: done: 0.170409 sec","job_status":"done","queue_duration_s":59.599288,"scheduling_latency_s":59.599331,"redis_calls":12,"redis_duration_s":0.003303,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000673,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002094,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000536,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.11873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000441397,"duration_s":0.170409,"completed_at":"2026-11-10T12:01:20.227Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027359,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 262.169680] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:20.301Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"194dbc2847a0efcc528f8005","created_at":"2026-11-10T12:00:20.480Z","trace_propagation_headers":{"sentry-trace":"6fb82f98c63f496fb522c81f4255b94e-a19c71b0ca304e9f","baggage":"sentry-trace_id=6fb82f98c63f496fb522c81f4255b94e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N96Y78W46XNW7TQ89SSRD","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:20.484Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-194dbc2847a0efcc528f8005: start","job_status":"start","queue_duration_s":59.816722,"scheduling_latency_s":59.816773} gitlab # [ 262.395617] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:20.527Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"194dbc2847a0efcc528f8005","created_at":"2026-11-10T12:00:20.480Z","trace_propagation_headers":{"sentry-trace":"6fb82f98c63f496fb522c81f4255b94e-a19c71b0ca304e9f","baggage":"sentry-trace_id=6fb82f98c63f496fb522c81f4255b94e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N96Y78W46XNW7TQ89SSRD","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:20.484Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-194dbc2847a0efcc528f8005: done: 0.226165 sec","job_status":"done","queue_duration_s":59.816722,"scheduling_latency_s":59.816773,"redis_calls":11,"redis_duration_s":0.002875,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000725,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00166,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00049,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.163,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.163,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.132014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000364013,"duration_s":0.226165,"completed_at":"2026-11-10T12:01:20.527Z","load_balancing_strategy":"primary","db_duration_s":0.078309,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 262.397451] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:20.528Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"9dc6ae544cda272cc620d6f5","created_at":"2026-11-10T12:00:20.486Z","trace_propagation_headers":{"sentry-trace":"6fb82f98c63f496fb522c81f4255b94e-a19c71b0ca304e9f","baggage":"sentry-trace_id=6fb82f98c63f496fb522c81f4255b94e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N96Y78W46XNW7TQ89SSRD","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:20.489Z","job_size_bytes":89,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-9dc6ae544cda272cc620d6f5: start","job_status":"start","queue_duration_s":60.038782,"scheduling_latency_s":60.0388} gitlab # [ 262.520905] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:20.651Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"9dc6ae544cda272cc620d6f5","created_at":"2026-11-10T12:00:20.486Z","trace_propagation_headers":{"sentry-trace":"6fb82f98c63f496fb522c81f4255b94e-a19c71b0ca304e9f","baggage":"sentry-trace_id=6fb82f98c63f496fb522c81f4255b94e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N96Y78W46XNW7TQ89SSRD","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:20.489Z","job_size_bytes":89,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-9dc6ae544cda272cc620d6f5: done: 0.122976 sec","job_status":"done","queue_duration_s":60.038782,"scheduling_latency_s":60.0388,"redis_calls":11,"redis_duration_s":0.003229,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00098,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001687,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000562,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.086,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.086,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000515429,"duration_s":0.122976,"completed_at":"2026-11-10T12:01:20.651Z","load_balancing_strategy":"primary","db_duration_s":0.089015,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 262.522670] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:20.653Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b377b4f54ff49ed544f9e576","created_at":"2026-11-10T12:00:27.518Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-9e5a15cd7495406e","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:27.522Z","job_size_bytes":185,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b377b4f54ff49ed544f9e576: start","job_status":"start","queue_duration_s":53.131369,"scheduling_latency_s":53.131387} gitlab # [ 262.596105] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:20.728Z","message":"changes: \"0000000000000000000000000000000000000000 bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 refs/heads/master\\n\""} gitlab # [ 262.649902] gitaly[919]: time="2026-11-10T12:01:20.781Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:01:30.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:20.770" grpc.time_ms=10.563 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.667386] gitaly[919]: time="2026-11-10T12:01:20.799Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T12:01:30.028" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:01:20.789" grpc.time_ms=10.264 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.742503] gitaly[919]: time="2026-11-10T12:01:20.874Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-10T12:01:30.102" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:20.862" grpc.time_ms=11.823 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.798428] gitaly[919]: time="2026-11-10T12:01:20.930Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:20.929" grpc.time_ms=1.167 pid=919 span.kind=server system=grpc gitlab # [ 262.836980] gitaly[919]: time="2026-11-10T12:01:20.969Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:20.968" grpc.time_ms=0.266 pid=919 span.kind=server system=grpc gitlab # [ 262.856447] gitaly[919]: time="2026-11-10T12:01:20.988Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB diskcache=93283d38-0851-43bb-b5af-e748e7b7a107 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T12:01:30.130" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:01:20.891" pid=919 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 262.865752] gitaly[919]: time="2026-11-10T12:01:20.997Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=118 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5258 command.oublock=8 command.real_time_ms=88 command.spawn_token_fork_ms=0 command.system_time_ms=86 command.user_time_ms=32 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T12:01:30.130" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:01:20.891" grpc.time_ms=97.502 pid=919 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 262.886233] gitaly[919]: time="2026-11-10T12:01:21.018Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T12:01:31.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:01:21.018" grpc.time_ms=0.104 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.907886] gitaly[919]: time="2026-11-10T12:01:21.039Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=941 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-10T12:01:31.012" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:21.023" grpc.time_ms=16.126 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.989511] gitaly[919]: time="2026-11-10T12:01:21.121Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=339 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-11-10T12:01:51.011" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:21.111" grpc.time_ms=9.451 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.011549] gitaly[919]: time="2026-11-10T12:01:21.143Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T12:01:31.132" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:01:21.143" grpc.time_ms=0.457 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.023442] gitaly[919]: time="2026-11-10T12:01:21.155Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=340 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-10T12:01:31.137" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-11-10T12:01:21.147" grpc.time_ms=7.461 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa' gitlab # [ 263.047126] gitaly[919]: time="2026-11-10T12:01:21.179Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:01:31.162" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:21.172" grpc.time_ms=6.082 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.060937] gitaly[919]: time="2026-11-10T12:01:21.192Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:01:31.173" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:21.184" grpc.time_ms=8.772 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.087544] gitaly[919]: time="2026-11-10T12:01:21.219Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=960 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-10T12:01:51.102" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:21.202" grpc.time_ms=17.032 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.105556] nixbot[2744]: {"ts": "2026-11-10T12:01:21.236Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:40356 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa HTTP/1.1\" 200"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa', in 0.05 seconds) gitlab # [ 263.457871] gitaly[919]: time="2026-11-10T12:01:21.589Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-10T12:01:31.007" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:21.588" grpc.time_ms=1.539 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.489908] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:21.621Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b377b4f54ff49ed544f9e576","created_at":"2026-11-10T12:00:27.518Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-9e5a15cd7495406e","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:00:27.522Z","job_size_bytes":185,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b377b4f54ff49ed544f9e576: done: 0.967616 sec","job_status":"done","queue_duration_s":53.131369,"scheduling_latency_s":53.131387,"gitaly_calls":13,"gitaly_duration_s":0.272435,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.050021,"redis_read_bytes":2103,"redis_write_bytes":21749,"redis_cache_calls":15,"redis_cache_duration_s":0.004727,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.007121,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001791,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7837,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005119,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.019816,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.011447,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.109,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.171,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.192,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.109,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.525628,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001061308,"exclusive_lock_hold_duration_s":0.038548475,"duration_s":0.967616,"completed_at":"2026-11-10T12:01:21.621Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.153919,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 263.552515] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:21.623Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"afb978f91923d5bb9b2ccefc","created_at":"2026-11-10T12:00:32.641Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-27eaa6a8fc1747f6","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:32.647Z","job_size_bytes":65,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-afb978f91923d5bb9b2ccefc: start","job_status":"start","queue_duration_s":48.976886,"scheduling_latency_s":48.976921} gitlab # [ 263.581465] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:21.713Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"afb978f91923d5bb9b2ccefc","created_at":"2026-11-10T12:00:32.641Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-27eaa6a8fc1747f6","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:32.647Z","job_size_bytes":65,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-afb978f91923d5bb9b2ccefc: done: 0.089353 sec","job_status":"done","queue_duration_s":48.976886,"scheduling_latency_s":48.976921,"redis_calls":16,"redis_duration_s":0.004732,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002528,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001674,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00053,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000328813,"duration_s":0.089353,"completed_at":"2026-11-10T12:01:21.713Z","load_balancing_strategy":"primary","db_duration_s":0.020224,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 263.582827] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:21.714Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"3814de57a5e0a83c8255e22d","created_at":"2026-11-10T12:00:32.647Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-27eaa6a8fc1747f6","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:32.650Z","job_size_bytes":65,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-3814de57a5e0a83c8255e22d: start","job_status":"start","queue_duration_s":49.063976,"scheduling_latency_s":49.063992} gitlab # [ 263.670638] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:21.801Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"3814de57a5e0a83c8255e22d","created_at":"2026-11-10T12:00:32.647Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-27eaa6a8fc1747f6","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:32.650Z","job_size_bytes":65,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-3814de57a5e0a83c8255e22d: done: 0.087017 sec","job_status":"done","queue_duration_s":49.063976,"scheduling_latency_s":49.063992,"redis_calls":21,"redis_duration_s":0.005865,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002218,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002085,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001562,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000596165,"duration_s":0.087017,"completed_at":"2026-11-10T12:01:21.801Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030332,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.672812] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:21.804Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4d9cce55afe5828551a5fca4","created_at":"2026-11-10T12:00:32.736Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-27eaa6a8fc1747f6","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:32.740Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4d9cce55afe5828551a5fca4: start","job_status":"start","queue_duration_s":49.063707,"scheduling_latency_s":49.063754} gitlab # [ 263.917232] gitaly[919]: time="2026-11-10T12:01:22.049Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:22.048" grpc.time_ms=0.644 pid=919 span.kind=server system=grpc gitlab # [ 263.966567] gitaly[919]: time="2026-11-10T12:01:22.098Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:22.098" grpc.time_ms=0.458 pid=919 span.kind=server system=grpc gitlab # [ 264.007994] gitaly[919]: time="2026-11-10T12:01:22.140Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:22.139" grpc.time_ms=0.699 pid=919 span.kind=server system=grpc gitlab # [ 264.034654] gitaly[919]: time="2026-11-10T12:01:22.166Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 diskcache=81e565b7-4238-4326-8df2-5dd422fcf850 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T12:01:52.004" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:22.005" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.035183] gitaly[919]: time="2026-11-10T12:01:22.167Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=139 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7444 command.oublock=0 command.real_time_ms=153 command.spawn_token_fork_ms=1 command.system_time_ms=90 command.user_time_ms=49 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T12:01:52.004" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:22.005" grpc.time_ms=161.321 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa' gitlab # [ 264.143780] nixbot[2744]: {"ts": "2026-11-10T12:01:22.275Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:40362 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa HTTP/1.1\" 200"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa', in 0.04 seconds) gitlab # [ 264.197474] gitaly[919]: time="2026-11-10T12:01:22.329Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T12:01:52.227" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:22.328" grpc.time_ms=1.361 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.253311] gitaly[919]: time="2026-11-10T12:01:22.385Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9KXMY7S11R34EFRM8Y11 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:52.283" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:22.383" grpc.time_ms=1.106 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.266878] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:22.398Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4d9cce55afe5828551a5fca4","created_at":"2026-11-10T12:00:32.736Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-27eaa6a8fc1747f6","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:32.740Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2616,"job_status":"rescheduled","message":"PipelineProcessWorker JID-4d9cce55afe5828551a5fca4: rescheduled"} gitlab # [ 264.276740] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:22.408Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"4d9cce55afe5828551a5fca4","created_at":"2026-11-10T12:00:32.736Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-27eaa6a8fc1747f6","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T12:00:32.740Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4d9cce55afe5828551a5fca4: done: 0.60398 sec","job_status":"done","queue_duration_s":49.063707,"scheduling_latency_s":49.063754,"gitaly_calls":3,"gitaly_duration_s":0.192705,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.034619,"redis_read_bytes":460,"redis_write_bytes":29291,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.0017,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001825,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.012108,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.005422,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15578,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.007391,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000835,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000876,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002762,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.069,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.092,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.098,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.035,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.301319,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.534592246,"duration_s":0.60398,"completed_at":"2026-11-10T12:01:22.408Z","load_balancing_strategy":"primary","db_duration_s":0.08545,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 264.353160] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:22.484Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 6279a8528aa5cc381d8ddfcd046a8d644342b5aa refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"c9fbc48258752e0aab22a8bc","created_at":"2026-11-10T12:01:18.933Z","trace_propagation_headers":{"sentry-trace":"8a718e2f9b164862a1542874123a7a98-007cfed4225b4f38","baggage":"sentry-trace_id=8a718e2f9b164862a1542874123a7a98,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72NB10ANG9CDV34HT585920","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:18.936Z","job_size_bytes":185,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-c9fbc48258752e0aab22a8bc: start","job_status":"start","queue_duration_s":3.547643,"scheduling_latency_s":3.54769} gitlab # [ 264.372876] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:22.504Z","message":"changes: \"bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 6279a8528aa5cc381d8ddfcd046a8d644342b5aa refs/heads/master\\n\""} gitlab # [ 264.410584] gitaly[919]: time="2026-11-10T12:01:22.542Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:01:32.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:22.530" grpc.time_ms=12.315 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.439570] gitaly[919]: time="2026-11-10T12:01:22.571Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T12:01:32.040" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:01:22.560" grpc.time_ms=10.539 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.444428] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:22.576Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"e8c3566b773d0203fdebe672","created_at":"2026-11-10T12:01:22.574Z","trace_propagation_headers":{"sentry-trace":"8a718e2f9b164862a1542874123a7a98-955f9c3527ff46b3","baggage":"sentry-trace_id=8a718e2f9b164862a1542874123a7a98,sentry-release=Unknown"},"correlation_id":"01M72NB10ANG9CDV34HT585920","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"33dfa1f0fc72692d88242786","job_size_bytes":46,"pid":2616,"job_status":"deduplicated","message":"ProjectCacheWorker JID-e8c3566b773d0203fdebe672: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 264.469223] gitaly[919]: time="2026-11-10T12:01:22.601Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-10T12:01:32.070" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:22.590" grpc.time_ms=10.434 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.492145] gitaly[919]: time="2026-11-10T12:01:22.624Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-11-10T12:01:52.513" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:22.613" grpc.time_ms=10.462 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.528746] gitaly[919]: time="2026-11-10T12:01:22.660Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=345 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-10T12:01:32.132" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-11-10T12:01:22.652" grpc.time_ms=8.008 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.555395] gitaly[919]: time="2026-11-10T12:01:22.687Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:01:32.155" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:22.675" grpc.time_ms=11.655 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.666541] gitaly[919]: time="2026-11-10T12:01:22.798Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-10T12:01:32.276" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:22.797" grpc.time_ms=1.3 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.688632] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:22.819Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 6279a8528aa5cc381d8ddfcd046a8d644342b5aa refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"c9fbc48258752e0aab22a8bc","created_at":"2026-11-10T12:01:18.933Z","trace_propagation_headers":{"sentry-trace":"8a718e2f9b164862a1542874123a7a98-007cfed4225b4f38","baggage":"sentry-trace_id=8a718e2f9b164862a1542874123a7a98,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72NB10ANG9CDV34HT585920","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:18.936Z","job_size_bytes":185,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-c9fbc48258752e0aab22a8bc: done: 0.334759 sec","job_status":"done","queue_duration_s":3.547643,"scheduling_latency_s":3.54769,"gitaly_calls":7,"gitaly_duration_s":0.121331,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.028672,"redis_read_bytes":501,"redis_write_bytes":11941,"redis_cache_calls":9,"redis_cache_duration_s":0.003145,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001194,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5718,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005002,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.009921,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.00941,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.03,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.03,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.182008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000941181,"exclusive_lock_hold_duration_s":0.038802976,"duration_s":0.334759,"completed_at":"2026-11-10T12:01:22.819Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029885,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 264.704658] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:22.821Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"33dfa1f0fc72692d88242786","created_at":"2026-11-10T12:01:20.823Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-394bf3ab1b5e4527","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:20.840Z","job_size_bytes":46,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-33dfa1f0fc72692d88242786: start","job_status":"start","queue_duration_s":1.980771,"scheduling_latency_s":1.98083} gitlab # [ 264.754708] gitaly[919]: time="2026-11-10T12:01:22.878Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-11-10T18:01:22.875" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:01:22.876" grpc.time_ms=2.126 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.764172] gitaly[919]: time="2026-11-10T12:01:22.896Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=657 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-10T12:01:52.785" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:22.886" grpc.time_ms=9.621 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.798099] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:22.929Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"33dfa1f0fc72692d88242786","created_at":"2026-11-10T12:01:20.823Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-394bf3ab1b5e4527","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:20.840Z","job_size_bytes":46,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-33dfa1f0fc72692d88242786: done: 0.108021 sec","job_status":"done","queue_duration_s":1.980771,"scheduling_latency_s":1.98083,"gitaly_calls":2,"gitaly_duration_s":0.016727,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.006461,"redis_read_bytes":69,"redis_write_bytes":4888,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.00122,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000517,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001439,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002649,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000636,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072527,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000379377,"duration_s":0.108021,"completed_at":"2026-11-10T12:01:22.929Z","load_balancing_strategy":"primary","db_duration_s":0.010036,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 264.801618] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:22.932Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"349bec37b162da6ba49526d3","created_at":"2026-11-10T12:01:20.854Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-394bf3ab1b5e4527","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:20.858Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-349bec37b162da6ba49526d3: start","job_status":"start","queue_duration_s":2.073286,"scheduling_latency_s":2.073314} gitlab # [ 264.850877] gitaly[919]: time="2026-11-10T12:01:22.976Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB diskcache=5066a35d-0acf-494a-a821-15eee96c2f4f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T18:01:22.953" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:22.954" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.851671] gitaly[919]: time="2026-11-10T12:01:22.977Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1284 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T18:01:22.953" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:22.954" grpc.time_ms=22.353 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.877512] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:23.009Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"349bec37b162da6ba49526d3","created_at":"2026-11-10T12:01:20.854Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-394bf3ab1b5e4527","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:20.858Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-349bec37b162da6ba49526d3: done: 0.077342 sec","job_status":"done","queue_duration_s":2.073286,"scheduling_latency_s":2.073314,"gitaly_calls":1,"gitaly_duration_s":0.024968,"redis_calls":15,"redis_duration_s":0.0035900000000000003,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001306,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00064,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001195,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.017,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.055796198000000005,"duration_s":0.077342,"completed_at":"2026-11-10T12:01:23.009Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016689,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 264.918197] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:23.012Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"3b304bdb1d75d5dd76211dac","created_at":"2026-11-10T12:01:21.012Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-394bf3ab1b5e4527","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:21.015Z","job_size_bytes":23,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-3b304bdb1d75d5dd76211dac: start","job_status":"start","queue_duration_s":1.997104,"scheduling_latency_s":1.997121} gitlab # [ 264.919236] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:23.049Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"3b304bdb1d75d5dd76211dac","created_at":"2026-11-10T12:01:21.012Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-394bf3ab1b5e4527","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:21.015Z","job_size_bytes":23,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-3b304bdb1d75d5dd76211dac: done: 0.036758 sec","job_status":"done","queue_duration_s":1.997104,"scheduling_latency_s":1.997121,"gitaly_calls":1,"gitaly_duration_s":0.011486,"redis_calls":16,"redis_duration_s":0.002379,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000214,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000692,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001041,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000432,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036758,"completed_at":"2026-11-10T12:01:23.049Z","load_balancing_strategy":"primary","db_duration_s":0.00286,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 264.959680] gitaly[919]: time="2026-11-10T12:01:23.042Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-10T12:01:53.032" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:23.033" grpc.time_ms=9.033 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.960572] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:23.050Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"5c43d15d85ea401a7066e5a1","created_at":"2026-11-10T12:01:21.436Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-394bf3ab1b5e4527","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e019f8b6da60080fb11b77cf511e97709f8ba53ea07ff7aeb3331dff286a55b0","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:21.440Z","job_size_bytes":467,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-5c43d15d85ea401a7066e5a1: start","job_status":"start","queue_duration_s":1.609827,"scheduling_latency_s":1.609846} gitlab # [ 264.961527] gitaly[919]: time="2026-11-10T12:01:23.074Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:53.073" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:23.073" grpc.time_ms=0.674 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 264.973069] gitaly[919]: time="2026-11-10T12:01:23.085Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T12:01:33.005" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:23.076" grpc.time_ms=8.522 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.029535] gitaly[919]: time="2026-11-10T12:01:23.161Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T12:01:53.160" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:23.160" grpc.time_ms=0.831 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.033775] gitaly[919]: time="2026-11-10T12:01:23.165Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=49 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-10T12:01:53.163" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:23.163" grpc.time_ms=2.017 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.107619] gitaly[919]: time="2026-11-10T12:01:23.239Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:01:33.159" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:23.230" grpc.time_ms=9.448 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa' gitlab # [ 265.196494] nixbot[2744]: {"ts": "2026-11-10T12:01:23.327Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:40368 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa HTTP/1.1\" 200"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa', in 0.05 seconds) gitlab # [ 265.592273] gitaly[919]: time="2026-11-10T12:01:23.724Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=961 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-10T12:01:53.708" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:23.708" grpc.time_ms=15.291 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.625582] gitaly[919]: time="2026-11-10T12:01:23.743Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72N9ERN4CXTBA6BYQFWKKCB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-10T12:01:53.728" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:23.729" grpc.time_ms=14.143 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 265.650706] sidekiq-cluster[2616]: {"severity":"WARN","time":"2026-11-10T12:01:23.767Z","class":"CreatePipelineWorker","correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5","ref":"refs/heads/master"},"retry":0} gitlab # [ 265.659245] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:23.773Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"5c43d15d85ea401a7066e5a1","created_at":"2026-11-10T12:01:21.436Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-394bf3ab1b5e4527","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e019f8b6da60080fb11b77cf511e97709f8ba53ea07ff7aeb3331dff286a55b0","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:21.440Z","job_size_bytes":467,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-5c43d15d85ea401a7066e5a1: done: 0.723695 sec","job_status":"done","queue_duration_s":1.609827,"scheduling_latency_s":1.609846,"gitaly_calls":7,"gitaly_duration_s":0.07986,"redis_calls":51,"redis_duration_s":0.009682,"redis_read_bytes":169,"redis_write_bytes":4167,"redis_cache_calls":4,"redis_cache_duration_s":0.000876,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.003975,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001189,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.00273,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.591045,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000291936,"duration_s":0.723695,"completed_at":"2026-11-10T12:01:23.773Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043463,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 265.703729] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:23.775Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"25bddfe5-3b40-4b9f-a2fc-5773d049d7de\"}"],"class":"WebHookWorker","jid":"98b22139e127c82f005f077b","created_at":"2026-11-10T12:01:21.561Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-394bf3ab1b5e4527","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5d4ed6900516589c768c5db175c6ded484d504124770afd403546cb208e90425","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:21.567Z","job_size_bytes":1751,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-98b22139e127c82f005f077b: start","job_status":"start","queue_duration_s":2.208301,"scheduling_latency_s":2.208319} gitlab # [ 265.774560] nixbot[2744]: {"ts": "2026-11-10T12:01:23.906Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "e0c27fb6-01b4-48dc-8203-1ce11c83eefb", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5", "pr": null} gitlab # [ 265.784988] nixbot[2744]: {"ts": "2026-11-10T12:01:23.917Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:40370 - \"POST /webhooks/gitlab HTTP/1.1\" 202"} gitlab # [ 265.812483] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:23.943Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"25bddfe5-3b40-4b9f-a2fc-5773d049d7de\"}"],"class":"WebHookWorker","jid":"98b22139e127c82f005f077b","created_at":"2026-11-10T12:01:21.561Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-394bf3ab1b5e4527","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5d4ed6900516589c768c5db175c6ded484d504124770afd403546cb208e90425","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:21.567Z","job_size_bytes":1751,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-98b22139e127c82f005f077b: done: 0.168353 sec","job_status":"done","queue_duration_s":2.208301,"scheduling_latency_s":2.208319,"redis_calls":19,"redis_duration_s":0.0060669999999999995,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001169,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000668,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002998,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001232,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.07079139899997244,"cpu_s":0.08041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000525206,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.168353,"completed_at":"2026-11-10T12:01:23.943Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003386,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.860847] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:23.947Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"fced61e8136f1a7bb5c383f3","created_at":"2026-11-10T12:01:21.993Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:21.996Z","job_size_bytes":23,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-fced61e8136f1a7bb5c383f3: start","job_status":"start","queue_duration_s":1.950842,"scheduling_latency_s":1.950877} gitlab # [ 265.904873] gitaly[919]: time="2026-11-10T12:01:24.036Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB6FDT8M0P55FD1WGEBRK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:24.029" grpc.time_ms=7.368 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 265.917451] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:24 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 99 gitlab # [ 265.986158] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.117Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"fced61e8136f1a7bb5c383f3","created_at":"2026-11-10T12:01:21.993Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:21.996Z","job_size_bytes":23,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-fced61e8136f1a7bb5c383f3: done: 0.169674 sec","job_status":"done","queue_duration_s":1.950842,"scheduling_latency_s":1.950877,"redis_calls":10,"redis_duration_s":0.002781,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000571,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001653,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000557,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.076,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.036,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000335797,"duration_s":0.169674,"completed_at":"2026-11-10T12:01:24.117Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.081064,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 266.050379] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.182Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d7dab3da900c558eaa4dfe14","created_at":"2026-11-10T12:01:22.193Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.201Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d7dab3da900c558eaa4dfe14: start","job_status":"start","queue_duration_s":1.980757,"scheduling_latency_s":1.9808} gitlab # [ 266.076580] gitaly[919]: time="2026-11-10T12:01:24.208Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72NB6JMYV53V1WXK0DEJ3RK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:24.196" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 266.077336] gitaly[919]: time="2026-11-10T12:01:24.208Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB6JMYV53V1WXK0DEJ3RK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:24.196" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 266.078199] gitlab-workhorse[825]: time="2026-11-10T12:01:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 266.078965] gitaly[919]: time="2026-11-10T12:01:24.208Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB6JMYV53V1WXK0DEJ3RK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:24.196" grpc.time_ms=12.001 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 266.101806] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.233Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d7dab3da900c558eaa4dfe14","created_at":"2026-11-10T12:01:22.193Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.201Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d7dab3da900c558eaa4dfe14: done: 0.051577 sec","job_status":"done","queue_duration_s":1.980757,"scheduling_latency_s":1.9808,"redis_calls":10,"redis_duration_s":0.003029,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000616,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001842,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000571,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000551188,"duration_s":0.051577,"completed_at":"2026-11-10T12:01:24.233Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006104,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.148805] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.235Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"58a1c71467ee5cd03af9bc0b","created_at":"2026-11-10T12:01:22.215Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.227Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-58a1c71467ee5cd03af9bc0b: start","job_status":"start","queue_duration_s":2.007534,"scheduling_latency_s":2.007562} gitlab # [ 266.149800] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.262Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"58a1c71467ee5cd03af9bc0b","created_at":"2026-11-10T12:01:22.215Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.227Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-58a1c71467ee5cd03af9bc0b: done: 0.02688 sec","job_status":"done","queue_duration_s":2.007534,"scheduling_latency_s":2.007562,"redis_calls":10,"redis_duration_s":0.0014889999999999999,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000287,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000877,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000325,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000227124,"duration_s":0.02688,"completed_at":"2026-11-10T12:01:24.262Z","load_balancing_strategy":"primary","db_duration_s":0.003646,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa' gitlab # [ 266.195847] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.263Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"9e4a801683649556d82c5115","created_at":"2026-11-10T12:01:22.246Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.252Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-9e4a801683649556d82c5115: start","job_status":"start","queue_duration_s":2.011292,"scheduling_latency_s":2.011306} gitlab # [ 266.196751] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.288Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"9e4a801683649556d82c5115","created_at":"2026-11-10T12:01:22.246Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.252Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-9e4a801683649556d82c5115: done: 0.024543 sec","job_status":"done","queue_duration_s":2.011292,"scheduling_latency_s":2.011306,"redis_calls":13,"redis_duration_s":0.001725,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000398,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000282,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000213156,"duration_s":0.024543,"completed_at":"2026-11-10T12:01:24.288Z","load_balancing_strategy":"primary","db_duration_s":0.003873,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa', in 0.06 seconds) gitlab # [ 266.241325] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.289Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"cfb5136116851888a8744a57","created_at":"2026-11-10T12:01:22.253Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.261Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-cfb5136116851888a8744a57: start","job_status":"start","queue_duration_s":2.027565,"scheduling_latency_s":2.027579} gitlab # [ 266.241942] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.310Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"cfb5136116851888a8744a57","created_at":"2026-11-10T12:01:22.253Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.261Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-cfb5136116851888a8744a57: done: 0.021468 sec","job_status":"done","queue_duration_s":2.027565,"scheduling_latency_s":2.027579,"redis_calls":10,"redis_duration_s":0.001878,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001099,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000307,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000268191,"duration_s":0.021468,"completed_at":"2026-11-10T12:01:24.310Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003205,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.293714] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.311Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c3e553eded1dca4821a883ad","created_at":"2026-11-10T12:01:22.262Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.269Z","job_size_bytes":19,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c3e553eded1dca4821a883ad: start","job_status":"start","queue_duration_s":2.041884,"scheduling_latency_s":2.041898} gitlab # [ 266.294473] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.332Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c3e553eded1dca4821a883ad","created_at":"2026-11-10T12:01:22.262Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.269Z","job_size_bytes":19,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c3e553eded1dca4821a883ad: done: 0.020564 sec","job_status":"done","queue_duration_s":2.041884,"scheduling_latency_s":2.041898,"redis_calls":11,"redis_duration_s":0.001722,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000346,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000314,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000207568,"duration_s":0.020564,"completed_at":"2026-11-10T12:01:24.332Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00366,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.334281] nixbot[2744]: {"ts": "2026-11-10T12:01:24.397Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:40382 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa HTTP/1.1\" 200"} gitlab # [ 266.334961] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.333Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9ccf3a2b9b4a20185901f39e","created_at":"2026-11-10T12:01:22.272Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.279Z","job_size_bytes":66,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9ccf3a2b9b4a20185901f39e: start","job_status":"start","queue_duration_s":2.053716,"scheduling_latency_s":2.053729} gitlab # [ 266.342269] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.392Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"9ccf3a2b9b4a20185901f39e","created_at":"2026-11-10T12:01:22.272Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.279Z","job_size_bytes":66,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-9ccf3a2b9b4a20185901f39e: done: 0.059677 sec","job_status":"done","queue_duration_s":2.053716,"scheduling_latency_s":2.053729,"redis_calls":10,"redis_duration_s":0.002956,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000391,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00228,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000285,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000238019,"duration_s":0.059677,"completed_at":"2026-11-10T12:01:24.392Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.037841,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.419274] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.393Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"878e03df43d6f248051d17d6","created_at":"2026-11-10T12:01:22.280Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.285Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-878e03df43d6f248051d17d6: start","job_status":"start","queue_duration_s":2.10852,"scheduling_latency_s":2.108534} gitlab # [ 266.419924] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.427Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"878e03df43d6f248051d17d6","created_at":"2026-11-10T12:01:22.280Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.285Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-878e03df43d6f248051d17d6: done: 0.033553 sec","job_status":"done","queue_duration_s":2.10852,"scheduling_latency_s":2.108534,"redis_calls":10,"redis_duration_s":0.002989,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000314,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002375,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0003,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000245562,"duration_s":0.033553,"completed_at":"2026-11-10T12:01:24.427Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003678,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.466576] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.428Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"cf3a590acc9b883d98d6324d","created_at":"2026-11-10T12:01:22.294Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.300Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-cf3a590acc9b883d98d6324d: start","job_status":"start","queue_duration_s":2.128359,"scheduling_latency_s":2.128377} gitlab # [ 266.467492] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.455Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"cf3a590acc9b883d98d6324d","created_at":"2026-11-10T12:01:22.294Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.300Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-cf3a590acc9b883d98d6324d: done: 0.027179 sec","job_status":"done","queue_duration_s":2.128359,"scheduling_latency_s":2.128377,"redis_calls":10,"redis_duration_s":0.0013900000000000002,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000266,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000849,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000214552,"duration_s":0.027179,"completed_at":"2026-11-10T12:01:24.455Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006996,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.521163] gitaly[919]: time="2026-11-10T12:01:24.590Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 diskcache=cbcffdd4-461c-4ea2-ab98-b48ce88add95 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T18:01:24.580" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:24.580" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.521985] gitaly[919]: time="2026-11-10T12:01:24.591Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=314 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T18:01:24.580" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:24.580" grpc.time_ms=10.29 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.522782] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.456Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e5c0c6599377303c8d2cdf94","created_at":"2026-11-10T12:01:22.301Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.305Z","job_size_bytes":6,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e5c0c6599377303c8d2cdf94: start","job_status":"start","queue_duration_s":2.151787,"scheduling_latency_s":2.151802} gitlab # [ 266.527517] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.472Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e5c0c6599377303c8d2cdf94","created_at":"2026-11-10T12:01:22.301Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.305Z","job_size_bytes":6,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e5c0c6599377303c8d2cdf94: done: 0.015462 sec","job_status":"done","queue_duration_s":2.151787,"scheduling_latency_s":2.151802,"redis_calls":10,"redis_duration_s":0.0013800000000000002,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000836,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000236063,"duration_s":0.015462,"completed_at":"2026-11-10T12:01:24.472Z","load_balancing_strategy":"primary","db_duration_s":0.0014,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.592310] gitaly[919]: time="2026-11-10T12:01:24.618Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:54.617" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:24.617" grpc.time_ms=0.512 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.592953] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.473Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"5a6ecf2c11069d5ac7317a6e","created_at":"2026-11-10T12:01:22.305Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.308Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-5a6ecf2c11069d5ac7317a6e: start","job_status":"start","queue_duration_s":2.165057,"scheduling_latency_s":2.165071} gitlab # [ 266.593633] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.497Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"5a6ecf2c11069d5ac7317a6e","created_at":"2026-11-10T12:01:22.305Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.308Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-5a6ecf2c11069d5ac7317a6e: done: 0.02421 sec","job_status":"done","queue_duration_s":2.165057,"scheduling_latency_s":2.165071,"redis_calls":10,"redis_duration_s":0.001985,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000259,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001406,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00032,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000247238,"duration_s":0.02421,"completed_at":"2026-11-10T12:01:24.497Z","load_balancing_strategy":"primary","db_duration_s":0.004304,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.630619] gitaly[919]: time="2026-11-10T12:01:24.627Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T12:01:34.100" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:24.620" grpc.time_ms=6.112 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.672685] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.498Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"7c83dc1f844c49cfe66b572e","created_at":"2026-11-10T12:01:22.313Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.316Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-7c83dc1f844c49cfe66b572e: start","job_status":"start","queue_duration_s":2.181521,"scheduling_latency_s":2.181536} gitlab # [ 266.673478] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.542Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"7c83dc1f844c49cfe66b572e","created_at":"2026-11-10T12:01:22.313Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.316Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-7c83dc1f844c49cfe66b572e: done: 0.043765 sec","job_status":"done","queue_duration_s":2.181521,"scheduling_latency_s":2.181536,"redis_calls":14,"redis_duration_s":0.002585,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00032,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000977,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001288,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000298083,"duration_s":0.043765,"completed_at":"2026-11-10T12:01:24.542Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007916,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.695043] gitaly[919]: time="2026-11-10T12:01:24.649Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T12:01:54.648" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:24.648" grpc.time_ms=0.634 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.736652] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.543Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"cb2e7f4fa05a7584e89b1bfd","created_at":"2026-11-10T12:01:22.399Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.401Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-cb2e7f4fa05a7584e89b1bfd: start","job_status":"start","queue_duration_s":2.142216,"scheduling_latency_s":2.142232} gitlab # [ 266.737415] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.567Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"cb2e7f4fa05a7584e89b1bfd","created_at":"2026-11-10T12:01:22.399Z","trace_propagation_headers":{"sentry-trace":"c4ef0ace5a764abb8382fe89fd873236-d5641d8d17e441bd","baggage":"sentry-trace_id=c4ef0ace5a764abb8382fe89fd873236,sentry-release=Unknown"},"correlation_id":"01M72N9KXMY7S11R34EFRM8Y11","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.401Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-cb2e7f4fa05a7584e89b1bfd: done: 0.023805 sec","job_status":"done","queue_duration_s":2.142216,"scheduling_latency_s":2.142232,"redis_calls":8,"redis_duration_s":0.001235,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000356,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023805,"completed_at":"2026-11-10T12:01:24.567Z","load_balancing_strategy":"primary","db_duration_s":0.003663,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 266.788787] gitaly[919]: time="2026-11-10T12:01:24.653Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=80 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-10T12:01:54.651" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:24.651" grpc.time_ms=1.814 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.789409] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.568Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"756bc5f704ced5d140afac96","created_at":"2026-11-10T12:01:22.583Z","trace_propagation_headers":{"sentry-trace":"8a718e2f9b164862a1542874123a7a98-955f9c3527ff46b3","baggage":"sentry-trace_id=8a718e2f9b164862a1542874123a7a98,sentry-release=Unknown"},"correlation_id":"01M72NB10ANG9CDV34HT585920","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"349bec37b162da6ba49526d3","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.585Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-756bc5f704ced5d140afac96: start","job_status":"start","queue_duration_s":1.98258,"scheduling_latency_s":1.9826} gitlab # [ 266.790057] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.601Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"756bc5f704ced5d140afac96","created_at":"2026-11-10T12:01:22.583Z","trace_propagation_headers":{"sentry-trace":"8a718e2f9b164862a1542874123a7a98-955f9c3527ff46b3","baggage":"sentry-trace_id=8a718e2f9b164862a1542874123a7a98,sentry-release=Unknown"},"correlation_id":"01M72NB10ANG9CDV34HT585920","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"349bec37b162da6ba49526d3","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.585Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-756bc5f704ced5d140afac96: done: 0.032859 sec","job_status":"done","queue_duration_s":1.98258,"scheduling_latency_s":1.9826,"gitaly_calls":1,"gitaly_duration_s":0.012323,"redis_calls":12,"redis_duration_s":0.001937,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000349,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000831,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.020783368,"duration_s":0.032859,"completed_at":"2026-11-10T12:01:24.601Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003042,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.843763] gitaly[919]: time="2026-11-10T12:01:24.677Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:01:34.151" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:24.672" grpc.time_ms=5.446 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.844508] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:24.602Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"12a3b137e3f7f3868076fc95","created_at":"2026-11-10T12:01:22.758Z","trace_propagation_headers":{"sentry-trace":"8a718e2f9b164862a1542874123a7a98-955f9c3527ff46b3","baggage":"sentry-trace_id=8a718e2f9b164862a1542874123a7a98,sentry-release=Unknown"},"correlation_id":"01M72NB10ANG9CDV34HT585920","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a38541e769f5ab0588316ea9c1a2e25190b4206b9ccee5b15b2fd569af294f69","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.762Z","job_size_bytes":467,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-12a3b137e3f7f3868076fc95: start","job_status":"start","queue_duration_s":1.840174,"scheduling_latency_s":1.840188} gitlab # [ 266.892470] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 156 gitlab # [ 266.930112] gitaly[919]: time="2026-11-10T12:01:25.062Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=968 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-10T12:01:55.049" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:25.050" grpc.time_ms=11.796 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.945162] gitaly[919]: time="2026-11-10T12:01:25.077Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72NB10ANG9CDV34HT585920 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-10T12:01:55.065" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:25.065" grpc.time_ms=11.182 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 266.958164] sidekiq-cluster[2616]: {"severity":"WARN","time":"2026-11-10T12:01:25.090Z","class":"CreatePipelineWorker","correlation_id":"01M72NB10ANG9CDV34HT585920","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5","after":"6279a8528aa5cc381d8ddfcd046a8d644342b5aa","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"6279a8528aa5cc381d8ddfcd046a8d644342b5aa","ref":"refs/heads/master"},"retry":0} gitlab # [ 266.962942] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:25.094Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"12a3b137e3f7f3868076fc95","created_at":"2026-11-10T12:01:22.758Z","trace_propagation_headers":{"sentry-trace":"8a718e2f9b164862a1542874123a7a98-955f9c3527ff46b3","baggage":"sentry-trace_id=8a718e2f9b164862a1542874123a7a98,sentry-release=Unknown"},"correlation_id":"01M72NB10ANG9CDV34HT585920","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a38541e769f5ab0588316ea9c1a2e25190b4206b9ccee5b15b2fd569af294f69","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.762Z","job_size_bytes":467,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-12a3b137e3f7f3868076fc95: done: 0.492177 sec","job_status":"done","queue_duration_s":1.840174,"scheduling_latency_s":1.840188,"gitaly_calls":7,"gitaly_duration_s":0.052143,"redis_calls":33,"redis_duration_s":0.003358,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000408,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000721,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001565,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000664,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.42644,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.492177,"completed_at":"2026-11-10T12:01:25.094Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012321,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 266.967250] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:25.097Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"9e108418-b195-49df-8f36-c0f15997ddcc\"}"],"class":"WebHookWorker","jid":"3a837ced13892952dfbc4d80","created_at":"2026-11-10T12:01:22.781Z","trace_propagation_headers":{"sentry-trace":"8a718e2f9b164862a1542874123a7a98-955f9c3527ff46b3","baggage":"sentry-trace_id=8a718e2f9b164862a1542874123a7a98,sentry-release=Unknown"},"correlation_id":"01M72NB10ANG9CDV34HT585920","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:867366beb733f3ed664342593f6936b9a5f0f9793c6f1f2318bb2d69e6ee3557","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.783Z","job_size_bytes":1749,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-3a837ced13892952dfbc4d80: start","job_status":"start","queue_duration_s":2.313799,"scheduling_latency_s":2.313821} gitlab # [ 267.008750] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:25.132Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"9e108418-b195-49df-8f36-c0f15997ddcc\"}"],"class":"WebHookWorker","jid":"3a837ced13892952dfbc4d80","created_at":"2026-11-10T12:01:22.781Z","trace_propagation_headers":{"sentry-trace":"8a718e2f9b164862a1542874123a7a98-955f9c3527ff46b3","baggage":"sentry-trace_id=8a718e2f9b164862a1542874123a7a98,sentry-release=Unknown"},"correlation_id":"01M72NB10ANG9CDV34HT585920","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:867366beb733f3ed664342593f6936b9a5f0f9793c6f1f2318bb2d69e6ee3557","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.783Z","job_size_bytes":1749,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-3a837ced13892952dfbc4d80: done: 0.034929 sec","job_status":"done","queue_duration_s":2.313799,"scheduling_latency_s":2.313821,"redis_calls":14,"redis_duration_s":0.002159,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000427,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.0007,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.0052836319999869374,"cpu_s":0.020455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.034929,"completed_at":"2026-11-10T12:01:25.132Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00216,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.024894] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:25.133Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"c863118bf49849a75bfb2acb","created_at":"2026-11-10T12:01:22.909Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-5afb5ef3b14a4194","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.915Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-c863118bf49849a75bfb2acb: start","job_status":"start","queue_duration_s":2.217393,"scheduling_latency_s":2.217414} gitlab # [ 267.039717] nixbot[2744]: {"ts": "2026-11-10T12:01:25.122Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "2df49e5f-4ca6-4109-ae6d-f73b2493107e", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "6279a8528aa5cc381d8ddfcd046a8d644342b5aa", "pr": null} gitlab # [ 267.077806] nixbot[2744]: {"ts": "2026-11-10T12:01:25.124Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:40394 - \"POST /webhooks/gitlab HTTP/1.1\" 202"} gitlab # [ 267.079072] gitaly[919]: time="2026-11-10T12:01:25.117Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72NB7H4XP4MH21DBGTS45M8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:25.101" pid=919 remote_ip="2001:db8:1::1" request_sha=9569be0011c0257a6690e4c7286a27924366fcc1 response_bytes=338 span.kind=server system=grpc gitlab # [ 267.080346] gitlab-workhorse[825]: time="2026-11-10T12:01:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=338 gitlab # [ 267.080880] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 338 "" "git/2.55.0" 90 gitlab # [ 267.081743] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:25.166Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"c863118bf49849a75bfb2acb","created_at":"2026-11-10T12:01:22.909Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-5afb5ef3b14a4194","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:22.915Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-c863118bf49849a75bfb2acb: done: 0.033267 sec","job_status":"done","queue_duration_s":2.217393,"scheduling_latency_s":2.217414,"redis_calls":19,"redis_duration_s":0.0026839999999999998,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000646,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000345,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00122,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000473,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0054646619999999995,"duration_s":0.033267,"completed_at":"2026-11-10T12:01:25.166Z","load_balancing_strategy":"primary","db_duration_s":0.00517,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.097604] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:25.167Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","98b22139e127c82f005f077b"],"class":"WebHooks::LogExecutionWorker","jid":"b08dbbfb3b4470ebc2919f52","created_at":"2026-11-10T12:01:23.921Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-fddebc7c118943e6","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9bcb2be6de1956a3ad21b55aa7fa69d577e6b2d8482b649cb125b98eccc9c50a","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:23.928Z","job_size_bytes":2592,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-b08dbbfb3b4470ebc2919f52: start","job_status":"start","queue_duration_s":1.238659,"scheduling_latency_s":1.238674} gitlab # [ 267.150630] gitaly[919]: time="2026-11-10T12:01:25.117Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1191 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB7H4XP4MH21DBGTS45M8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:25.101" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 267.151456] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:25.199Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","98b22139e127c82f005f077b"],"class":"WebHooks::LogExecutionWorker","jid":"b08dbbfb3b4470ebc2919f52","created_at":"2026-11-10T12:01:23.921Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-fddebc7c118943e6","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9bcb2be6de1956a3ad21b55aa7fa69d577e6b2d8482b649cb125b98eccc9c50a","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:23.928Z","job_size_bytes":2592,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-b08dbbfb3b4470ebc2919f52: done: 0.032238 sec","job_status":"done","queue_duration_s":1.238659,"scheduling_latency_s":1.238674,"redis_calls":10,"redis_duration_s":0.001528,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000291,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000963,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000274,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022852,"duration_s":0.032238,"completed_at":"2026-11-10T12:01:25.199Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005872,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.170905] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:25.201Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","3a837ced13892952dfbc4d80"],"class":"WebHooks::LogExecutionWorker","jid":"03c0cb601f395ebb5f5a5e2a","created_at":"2026-11-10T12:01:25.125Z","trace_propagation_headers":{"sentry-trace":"8a718e2f9b164862a1542874123a7a98-d26ad3dd4f2b42a5","baggage":"sentry-trace_id=8a718e2f9b164862a1542874123a7a98,sentry-release=Unknown"},"correlation_id":"01M72NB10ANG9CDV34HT585920","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:71c14aacf5e8d8ae88428f015f84a8a266a7750dd69e09b2239468c49ba30e38","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.127Z","job_size_bytes":2592,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-03c0cb601f395ebb5f5a5e2a: start","job_status":"start","queue_duration_s":0.073479,"scheduling_latency_s":0.073495} gitlab # [ 267.216937] gitaly[919]: time="2026-11-10T12:01:25.117Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1191 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB7H4XP4MH21DBGTS45M8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:25.101" grpc.time_ms=16.585 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 267.217692] nixbot[2744]: {"ts": "2026-11-10T12:01:25.290Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 267.218107] nixbot[2744]: {"ts": "2026-11-10T12:01:25.291Z", "level": "info", "logger": "nixbot.after_build", "msg": "reused build: replaying effect statuses", "build_id": 1, "sha": "bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa' gitlab # [ 267.218636] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:25.257Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","3a837ced13892952dfbc4d80"],"class":"WebHooks::LogExecutionWorker","jid":"03c0cb601f395ebb5f5a5e2a","created_at":"2026-11-10T12:01:25.125Z","trace_propagation_headers":{"sentry-trace":"8a718e2f9b164862a1542874123a7a98-d26ad3dd4f2b42a5","baggage":"sentry-trace_id=8a718e2f9b164862a1542874123a7a98,sentry-release=Unknown"},"correlation_id":"01M72NB10ANG9CDV34HT585920","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:71c14aacf5e8d8ae88428f015f84a8a266a7750dd69e09b2239468c49ba30e38","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.127Z","job_size_bytes":2592,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-03c0cb601f395ebb5f5a5e2a: done: 0.055786 sec","job_status":"done","queue_duration_s":0.073479,"scheduling_latency_s":0.073495,"redis_calls":7,"redis_duration_s":0.001232,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00029,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.039,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.039,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.055786,"completed_at":"2026-11-10T12:01:25.256Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.260878] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:25.258Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"2c52f39cfe446bb87a357d86","created_at":"2026-11-10T12:01:25.156Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-058888ec725d4df4","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.160Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-2c52f39cfe446bb87a357d86: start","job_status":"start","queue_duration_s":0.097752,"scheduling_latency_s":0.097775} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa', in 0.04 seconds) gitlab # [ 267.261725] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:25.388Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"2c52f39cfe446bb87a357d86","created_at":"2026-11-10T12:01:25.156Z","trace_propagation_headers":{"sentry-trace":"571341f0eb6f47bd97368dbac47d74cb-058888ec725d4df4","baggage":"sentry-trace_id=571341f0eb6f47bd97368dbac47d74cb,sentry-release=Unknown"},"correlation_id":"01M72N9ERN4CXTBA6BYQFWKKCB","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.160Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-2c52f39cfe446bb87a357d86: done: 0.129899 sec","job_status":"done","queue_duration_s":0.097752,"scheduling_latency_s":0.097775,"redis_calls":11,"redis_duration_s":0.002335,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000848,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001223,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000264,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.057,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000230197,"duration_s":0.129899,"completed_at":"2026-11-10T12:01:25.388Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.116247,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.301608] gitaly[919]: time="2026-11-10T12:01:25.366Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB7SYNMM165AN2X4VWZP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:25.359" grpc.time_ms=6.549 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 267.302229] gitaly[919]: time="2026-11-10T12:01:25.408Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72NB7VQ56P5YMXMYF783QS3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:25.401" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 267.347082] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 56 gitlab # [ 267.347690] gitlab-workhorse[825]: time="2026-11-10T12:01:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 267.348178] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 41 gitlab # [ 267.348662] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 36 gitlab # [ 267.349441] nixbot[2744]: {"ts": "2026-11-10T12:01:25.439Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:40396 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa HTTP/1.1\" 200"} gitlab # [ 267.349931] gitaly[919]: time="2026-11-10T12:01:25.408Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB7VQ56P5YMXMYF783QS3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:25.401" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 267.350651] gitaly[919]: time="2026-11-10T12:01:25.408Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB7VQ56P5YMXMYF783QS3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:25.401" grpc.time_ms=7.37 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 267.351248] gitaly[919]: time="2026-11-10T12:01:25.475Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB7XYMN4N0H8GDJEJ4E7Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:25.469" grpc.time_ms=5.879 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 267.396501] gitaly[919]: time="2026-11-10T12:01:25.528Z" level=info msg="finished unary call with code OK" catfile.duration_ms=178 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=178 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB7SYNMM165AN2S7EGD48 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:55.249" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:25.349" grpc.time_ms=178.498 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.400346] gitaly[919]: time="2026-11-10T12:01:25.532Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72NB7Z4ETZ9DASY4JV176TG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:25.520" pid=919 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 267.401140] gitaly[919]: time="2026-11-10T12:01:25.532Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB7Z4ETZ9DASY4JV176TG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:25.520" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 267.401906] gitaly[919]: time="2026-11-10T12:01:25.533Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB7Z4ETZ9DASY4JV176TG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:25.520" grpc.time_ms=12.27 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 267.402773] gitlab-workhorse[825]: time="2026-11-10T12:01:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 267.442947] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 56 gitlab # [ 267.454630] gitaly[919]: time="2026-11-10T12:01:25.586Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB7SYNMM165AN2S7EGD48 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:01:35.249" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:25.580" grpc.time_ms=6.694 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.477788] gitaly[919]: time="2026-11-10T12:01:25.609Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB81X6DX9ZB8PWC9JYDR7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:55.508" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:25.609" grpc.time_ms=0.581 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.496104] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:25.628Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0fb362e894fce00d4588932a","created_at":"2026-11-10T12:01:25.623Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-438b23f6b46e4cd6","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.626Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0fb362e894fce00d4588932a: start","job_status":"start","queue_duration_s":0.001566,"scheduling_latency_s":0.001597} gitlab # [ 267.510475] gitaly[919]: time="2026-11-10T12:01:25.635Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB81X6DX9ZB8PWC9JYDR7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-10T12:01:55.529" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:25.629" grpc.time_ms=6.132 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.523329] gitaly[919]: time="2026-11-10T12:01:25.649Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB81X6DX9ZB8PWC9JYDR7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:01:35.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:25.643" grpc.time_ms=6.016 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.536064] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:25 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 357 gitlab # [ 267.544443] gitaly[919]: time="2026-11-10T12:01:25.676Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72NB81X6DX9ZB8PWC9JYDR7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T12:01:35.035" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:01:25.676" grpc.time_ms=0.28 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.564320] gitaly[919]: time="2026-11-10T12:01:25.696Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB81X6DX9ZB8PWC9JYDR7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T12:01:35.050" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:25.690" grpc.time_ms=5.959 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.594582] gitaly[919]: time="2026-11-10T12:01:25.726Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB85B2DJWF36H0JQJM6HB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:55.625" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:25.726" grpc.time_ms=0.592 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.597115] gitaly[919]: time="2026-11-10T12:01:25.729Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72NB7SYNMM165AN2S7EGD48 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:25.728" grpc.time_ms=0.311 pid=919 span.kind=server system=grpc gitlab # [ 267.622682] gitaly[919]: time="2026-11-10T12:01:25.754Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72NB81X6DX9ZB8PWC9JYDR7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:25.754" grpc.time_ms=0.325 pid=919 span.kind=server system=grpc gitlab # [ 267.635470] gitaly[919]: time="2026-11-10T12:01:25.767Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB85B2DJWF36H0JQJM6HB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:01:35.032" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:25.752" grpc.time_ms=14.78 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.641379] gitaly[919]: time="2026-11-10T12:01:25.773Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72NB7SYNMM165AN2S7EGD48 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:25.772" grpc.time_ms=0.691 pid=919 span.kind=server system=grpc gitlab # [ 267.670888] gitaly[919]: time="2026-11-10T12:01:25.803Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72NB81X6DX9ZB8PWC9JYDR7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:25.802" grpc.time_ms=0.303 pid=919 span.kind=server system=grpc gitlab # [ 267.677515] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:25 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 133 gitlab # [ 267.683853] gitaly[919]: time="2026-11-10T12:01:25.815Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72NB7SYNMM165AN2S7EGD48 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:25.815" grpc.time_ms=0.3 pid=919 span.kind=server system=grpc gitlab # [ 267.696157] gitaly[919]: time="2026-11-10T12:01:25.828Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72NB81X6DX9ZB8PWC9JYDR7 diskcache=da59a457-c9f6-46a2-8462-f029da5706b0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T12:01:35.068" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:01:25.708" pid=919 ref=refs/keep-around/6279a8528aa5cc381d8ddfcd046a8d644342b5aa remote_ip="2001:db8:1::1" revision=6279a8528aa5cc381d8ddfcd046a8d644342b5aa span.kind=server system=grpc username=root gitlab # [ 267.707250] gitaly[919]: time="2026-11-10T12:01:25.839Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=59 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5123 command.oublock=8 command.real_time_ms=106 command.spawn_token_fork_ms=1 command.system_time_ms=45 command.user_time_ms=14 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB81X6DX9ZB8PWC9JYDR7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T12:01:35.068" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T12:01:25.708" grpc.time_ms=130.428 pid=919 ref=refs/keep-around/6279a8528aa5cc381d8ddfcd046a8d644342b5aa remote_ip="2001:db8:1::1" revision=6279a8528aa5cc381d8ddfcd046a8d644342b5aa span.kind=server system=grpc username=root gitlab # [ 267.709168] gitaly[919]: time="2026-11-10T12:01:25.841Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72NB7SYNMM165AN2S7EGD48 diskcache=57d0362a-c615-472b-a7f1-ba41303adedc grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T12:01:55.708" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:25.708" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.711177] gitaly[919]: time="2026-11-10T12:01:25.841Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=104 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7276 command.oublock=0 command.real_time_ms=116 command.spawn_token_fork_ms=0 command.system_time_ms=71 command.user_time_ms=33 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB7SYNMM165AN2S7EGD48 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T12:01:55.708" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:25.708" grpc.time_ms=132.591 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.732758] gitaly[919]: time="2026-11-10T12:01:25.864Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB89S33MT5CWP07430JJW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:55.763" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:25.864" grpc.time_ms=0.553 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.755187] gitaly[919]: time="2026-11-10T12:01:25.887Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB7SYNMM165AN2S7EGD48 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T12:01:55.886" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:25.886" grpc.time_ms=0.54 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.755638] gitaly[919]: time="2026-11-10T12:01:25.887Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB81X6DX9ZB8PWC9JYDR7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:55.787" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:25.887" grpc.time_ms=0.214 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.782031] gitaly[919]: time="2026-11-10T12:01:25.914Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB7SYNMM165AN2S7EGD48 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:55.013" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:25.913" grpc.time_ms=0.499 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.796743] gitaly[919]: time="2026-11-10T12:01:25.928Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB89S33MT5CWP07430JJW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:01:35.193" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:25.913" grpc.time_ms=14.932 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 267.834088] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:25.965Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0e7de02b7246323859f3eaf6","created_at":"2026-11-10T12:01:25.963Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"0fb362e894fce00d4588932a","job_size_bytes":3,"pid":2616,"job_status":"deduplicated","message":"PipelineProcessWorker JID-0e7de02b7246323859f3eaf6: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 267.852789] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:25.984Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0fb362e894fce00d4588932a","created_at":"2026-11-10T12:01:25.623Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-438b23f6b46e4cd6","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.626Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0fb362e894fce00d4588932a: done: 0.356828 sec","job_status":"done","queue_duration_s":0.001566,"scheduling_latency_s":0.001597,"gitaly_calls":3,"gitaly_duration_s":0.148043,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.041469,"redis_read_bytes":221,"redis_write_bytes":17174,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.0028,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00064,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":20,"redis_queues_duration_s":0.002593,"redis_queues_read_bytes":21,"redis_queues_write_bytes":11227,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.019257,"redis_queues_metadata_read_bytes":108,"redis_queues_metadata_write_bytes":3339,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.001199,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000715,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.011465,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":18,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.007,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.025,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.121755,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.253328566,"duration_s":0.356828,"completed_at":"2026-11-10T12:01:25.984Z","load_balancing_strategy":"primary","db_duration_s":0.038872,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 267.903087] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.033Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"84b7f8644ff7d78813b1138c","created_at":"2026-11-10T12:01:25.695Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.697Z","job_size_bytes":23,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-84b7f8644ff7d78813b1138c: start","job_status":"start","queue_duration_s":0.335974,"scheduling_latency_s":0.336012} gitlab # [ 267.918718] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6279a8528aa5cc381d8ddfcd046a8d644342b5aa HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 485 gitlab # [ 267.930129] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bdc26eb0a0f4d5fb6242dbd9116e42b3c96ba2a5 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 244 gitlab # [ 267.934632] nixbot[2744]: {"ts": "2026-11-10T12:01:26.066Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-6279a8528aa5-4"} gitlab # [ 267.960705] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.092Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"84b7f8644ff7d78813b1138c","created_at":"2026-11-10T12:01:25.695Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.697Z","job_size_bytes":23,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-84b7f8644ff7d78813b1138c: done: 0.058873 sec","job_status":"done","queue_duration_s":0.335974,"scheduling_latency_s":0.336012,"redis_calls":7,"redis_duration_s":0.004475,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004017,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000458,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.058873,"completed_at":"2026-11-10T12:01:26.092Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012177,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 268.001147] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.132Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"2ea2f09d88ce7a06141df2fc","created_at":"2026-11-10T12:01:25.846Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.848Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-2ea2f09d88ce7a06141df2fc: start","job_status":"start","queue_duration_s":0.284743,"scheduling_latency_s":0.284784} gitlab # [ 268.027289] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.159Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"2ea2f09d88ce7a06141df2fc","created_at":"2026-11-10T12:01:25.846Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.848Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-2ea2f09d88ce7a06141df2fc: done: 0.02634 sec","job_status":"done","queue_duration_s":0.284743,"scheduling_latency_s":0.284784,"redis_calls":7,"redis_duration_s":0.001619,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000752,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01464,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02634,"completed_at":"2026-11-10T12:01:26.159Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00439,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.029063] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.160Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"55c75b0ddaee0bee00d90b87","created_at":"2026-11-10T12:01:25.855Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.857Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-55c75b0ddaee0bee00d90b87: start","job_status":"start","queue_duration_s":0.302835,"scheduling_latency_s":0.302852} gitlab # [ 268.067822] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.184Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"55c75b0ddaee0bee00d90b87","created_at":"2026-11-10T12:01:25.855Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.857Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-55c75b0ddaee0bee00d90b87: done: 0.024062 sec","job_status":"done","queue_duration_s":0.302835,"scheduling_latency_s":0.302852,"redis_calls":7,"redis_duration_s":0.00126,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000389,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024062,"completed_at":"2026-11-10T12:01:26.184Z","load_balancing_strategy":"primary","db_duration_s":0.003784,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.109992] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.185Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"741c75de4bd5cb204bf7bf42","created_at":"2026-11-10T12:01:25.864Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.866Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-741c75de4bd5cb204bf7bf42: start","job_status":"start","queue_duration_s":0.318907,"scheduling_latency_s":0.318928} gitlab # [ 268.110627] nixbot[2744]: {"ts": "2026-11-10T12:01:26.171Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-6279a8528aa5-4", "argv0": "bwrap"} gitlab # [ 268.111201] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.210Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"741c75de4bd5cb204bf7bf42","created_at":"2026-11-10T12:01:25.864Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.866Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-741c75de4bd5cb204bf7bf42: done: 0.02449 sec","job_status":"done","queue_duration_s":0.318907,"scheduling_latency_s":0.318928,"redis_calls":10,"redis_duration_s":0.005282,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.004125,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02449,"completed_at":"2026-11-10T12:01:26.210Z","load_balancing_strategy":"primary","db_duration_s":0.003165,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.167869] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.211Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ee8ef67e82b760804f22dcf5","created_at":"2026-11-10T12:01:25.867Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.869Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ee8ef67e82b760804f22dcf5: start","job_status":"start","queue_duration_s":0.341909,"scheduling_latency_s":0.341924} gitlab # [ 268.168749] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.228Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ee8ef67e82b760804f22dcf5","created_at":"2026-11-10T12:01:25.867Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.869Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ee8ef67e82b760804f22dcf5: done: 0.017381 sec","job_status":"done","queue_duration_s":0.341909,"scheduling_latency_s":0.341924,"redis_calls":7,"redis_duration_s":0.000992,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000732,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011267,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017381,"completed_at":"2026-11-10T12:01:26.228Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002689,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.184294] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.229Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9dff410524a77d8a4a38ed2d","created_at":"2026-11-10T12:01:25.869Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.871Z","job_size_bytes":19,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9dff410524a77d8a4a38ed2d: start","job_status":"start","queue_duration_s":0.35845,"scheduling_latency_s":0.358465} gitlab # [ 268.227379] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.245Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9dff410524a77d8a4a38ed2d","created_at":"2026-11-10T12:01:25.869Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.871Z","job_size_bytes":19,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9dff410524a77d8a4a38ed2d: done: 0.016333 sec","job_status":"done","queue_duration_s":0.35845,"scheduling_latency_s":0.358465,"redis_calls":8,"redis_duration_s":0.001357,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000476,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016333,"completed_at":"2026-11-10T12:01:26.245Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002756,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.235951] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.246Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"298edf07add5b403e54ac86d","created_at":"2026-11-10T12:01:25.871Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.872Z","job_size_bytes":66,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-298edf07add5b403e54ac86d: start","job_status":"start","queue_duration_s":0.373865,"scheduling_latency_s":0.37388} gitlab # [ 268.289390] gitaly[919]: time="2026-11-10T12:01:26.389Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=102 command.spawn_token_fork_ms=97 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB8JGXZ0RYBYB0WF6JXR7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:26.285" grpc.time_ms=103.787 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 268.291430] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 293 gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa' gitlab # [ 268.294091] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.261Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"298edf07add5b403e54ac86d","created_at":"2026-11-10T12:01:25.871Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.872Z","job_size_bytes":66,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-298edf07add5b403e54ac86d: done: 0.014334 sec","job_status":"done","queue_duration_s":0.373865,"scheduling_latency_s":0.37388,"redis_calls":7,"redis_duration_s":0.000943,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000687,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014334,"completed_at":"2026-11-10T12:01:26.261Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001697,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.342194] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.262Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"688e230f07dc14d78d666f07","created_at":"2026-11-10T12:01:25.873Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.874Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-688e230f07dc14d78d666f07: start","job_status":"start","queue_duration_s":0.387413,"scheduling_latency_s":0.387427} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa', in 0.06 seconds) (finished: subtest: push delivers the webhook, in 55.16 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses gitlab # [ 268.356935] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.279Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"688e230f07dc14d78d666f07","created_at":"2026-11-10T12:01:25.873Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.874Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-688e230f07dc14d78d666f07: done: 0.017305 sec","job_status":"done","queue_duration_s":0.387413,"scheduling_latency_s":0.387427,"redis_calls":7,"redis_duration_s":0.001334,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001047,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000287,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017305,"completed_at":"2026-11-10T12:01:26.279Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003815,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.395231] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.280Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"399d3260fb18a4dad534ccf7","created_at":"2026-11-10T12:01:25.875Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.875Z","job_size_bytes":6,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-399d3260fb18a4dad534ccf7: start","job_status":"start","queue_duration_s":0.404681,"scheduling_latency_s":0.404696} gitlab # [ 268.409549] gitlab-workhorse[825]: time="2026-11-10T12:01:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 268.411307] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 51 gitlab # [ 268.413560] gitaly[919]: time="2026-11-10T12:01:26.442Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72NB8VQGN6ZEBHKZ5ZRHD0X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:26.431" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 268.422799] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.290Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"399d3260fb18a4dad534ccf7","created_at":"2026-11-10T12:01:25.875Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.875Z","job_size_bytes":6,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-399d3260fb18a4dad534ccf7: done: 0.009659 sec","job_status":"done","queue_duration_s":0.404681,"scheduling_latency_s":0.404696,"redis_calls":7,"redis_duration_s":0.001069,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000734,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000335,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009659,"completed_at":"2026-11-10T12:01:26.290Z","load_balancing_strategy":"primary","db_duration_s":0.000494,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.460685] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.291Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"db1d6e3b4e393d694beac501","created_at":"2026-11-10T12:01:25.876Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.877Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-db1d6e3b4e393d694beac501: start","job_status":"start","queue_duration_s":0.414126,"scheduling_latency_s":0.414139} gitlab # [ 268.462554] nixbot[2744]: {"ts": "2026-11-10T12:01:26.492Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:40410 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=6279a8528aa5cc381d8ddfcd046a8d644342b5aa HTTP/1.1\" 200"} gitlab # [ 268.465817] nix-daemon[2905]: accepted connection from pid 3749, user nixbot gitlab # [ 268.466809] gitaly[919]: time="2026-11-10T12:01:26.443Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB8VQGN6ZEBHKZ5ZRHD0X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:26.431" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 268.467588] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.309Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"db1d6e3b4e393d694beac501","created_at":"2026-11-10T12:01:25.876Z","trace_propagation_headers":{"sentry-trace":"13b9de30456245ac93ea6d671a4e2e89-5c87e059898d4b93","baggage":"sentry-trace_id=13b9de30456245ac93ea6d671a4e2e89,sentry-release=Unknown"},"correlation_id":"01M72NB7SYNMM165AN2S7EGD48","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.877Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-db1d6e3b4e393d694beac501: done: 0.018077 sec","job_status":"done","queue_duration_s":0.414126,"scheduling_latency_s":0.414139,"redis_calls":7,"redis_duration_s":0.003603,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002662,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018077,"completed_at":"2026-11-10T12:01:26.309Z","load_balancing_strategy":"primary","db_duration_s":0.002588,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.487547] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.312Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"98a2b7389c8360dfb7a9180f","created_at":"2026-11-10T12:01:25.903Z","trace_propagation_headers":{"sentry-trace":"b2ba98fa6e33457eb17d99cee943d6e8-74b3115e391b453e","baggage":"sentry-trace_id=b2ba98fa6e33457eb17d99cee943d6e8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72NB81X6DX9ZB8PWC9JYDR7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.907Z","job_size_bytes":65,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-98a2b7389c8360dfb7a9180f: start","job_status":"start","queue_duration_s":0.404753,"scheduling_latency_s":0.404769} gitlab # [ 268.502690] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 92 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses, in 0.18 seconds) gitlab # [ 268.507882] gitaly[919]: time="2026-11-10T12:01:26.443Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB8VQGN6ZEBHKZ5ZRHD0X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:26.431" grpc.time_ms=11.468 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 268.553768] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.335Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"98a2b7389c8360dfb7a9180f","created_at":"2026-11-10T12:01:25.903Z","trace_propagation_headers":{"sentry-trace":"b2ba98fa6e33457eb17d99cee943d6e8-74b3115e391b453e","baggage":"sentry-trace_id=b2ba98fa6e33457eb17d99cee943d6e8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72NB81X6DX9ZB8PWC9JYDR7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.907Z","job_size_bytes":65,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-98a2b7389c8360dfb7a9180f: done: 0.02303 sec","job_status":"done","queue_duration_s":0.404753,"scheduling_latency_s":0.404769,"redis_calls":7,"redis_duration_s":0.00317,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002624,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000546,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02303,"completed_at":"2026-11-10T12:01:26.335Z","load_balancing_strategy":"primary","db_duration_s":0.004419,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 268.554793] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.336Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"df148f2b165567b227f9bbeb","created_at":"2026-11-10T12:01:25.908Z","trace_propagation_headers":{"sentry-trace":"b2ba98fa6e33457eb17d99cee943d6e8-74b3115e391b453e","baggage":"sentry-trace_id=b2ba98fa6e33457eb17d99cee943d6e8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72NB81X6DX9ZB8PWC9JYDR7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.910Z","job_size_bytes":65,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-df148f2b165567b227f9bbeb: start","job_status":"start","queue_duration_s":0.426381,"scheduling_latency_s":0.426397} gitlab # [ 268.600638] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:26 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 85 gitlab # [ 268.601409] gitlab-workhorse[825]: time="2026-11-10T12:01:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 268.601946] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 49 gitlab # [ 268.602293] gitaly[919]: time="2026-11-10T12:01:26.474Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72NB89S33MT5CWP07430JJW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:26.474" grpc.time_ms=0.323 pid=919 span.kind=server system=grpc gitlab # [ 268.603002] gitaly[919]: time="2026-11-10T12:01:26.518Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72NB89S33MT5CWP07430JJW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:26.517" grpc.time_ms=0.556 pid=919 span.kind=server system=grpc gitlab # [ 268.603545] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.378Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"df148f2b165567b227f9bbeb","created_at":"2026-11-10T12:01:25.908Z","trace_propagation_headers":{"sentry-trace":"b2ba98fa6e33457eb17d99cee943d6e8-74b3115e391b453e","baggage":"sentry-trace_id=b2ba98fa6e33457eb17d99cee943d6e8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72NB81X6DX9ZB8PWC9JYDR7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.910Z","job_size_bytes":65,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-df148f2b165567b227f9bbeb: done: 0.041466 sec","job_status":"done","queue_duration_s":0.426381,"scheduling_latency_s":0.426397,"redis_calls":10,"redis_duration_s":0.00729,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001633,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.005657,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041466,"completed_at":"2026-11-10T12:01:26.378Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010139,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.648216] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.379Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"7d5b0966e4c5658edb8368a0","created_at":"2026-11-10T12:01:25.969Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-c9c49feb526b44ca","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.981Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7d5b0966e4c5658edb8368a0: start","job_status":"start","queue_duration_s":0.397159,"scheduling_latency_s":0.397176} gitlab # [ 268.649399] gitaly[919]: time="2026-11-10T12:01:26.581Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72NB89S33MT5CWP07430JJW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T12:01:26.579" grpc.time_ms=0.44 pid=919 span.kind=server system=grpc gitlab # [ 268.656403] gitaly[919]: time="2026-11-10T12:01:26.584Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M72NB89S33MT5CWP07430JJW forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T12:01:56.439" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:26.440" operations="[repack_refs]" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 268.699171] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.773Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"7d5b0966e4c5658edb8368a0","created_at":"2026-11-10T12:01:25.969Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-c9c49feb526b44ca","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.981Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7d5b0966e4c5658edb8368a0: done: 0.393962 sec","job_status":"done","queue_duration_s":0.397159,"scheduling_latency_s":0.397176,"gitaly_calls":3,"gitaly_duration_s":0.263798,"redis_calls":63,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.012862,"redis_read_bytes":142,"redis_write_bytes":16720,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.001008,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000646,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":20,"redis_queues_duration_s":0.001948,"redis_queues_read_bytes":22,"redis_queues_write_bytes":11226,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.002985,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000187,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000397,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.004683,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.09524,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.3621843,"duration_s":0.393962,"completed_at":"2026-11-10T12:01:26.773Z","load_balancing_strategy":"primary","db_duration_s":0.023561,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 268.744153] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.774Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"2f17a73233ca8a0b3eb2afcb","created_at":"2026-11-10T12:01:25.967Z","trace_propagation_headers":{"sentry-trace":"b2ba98fa6e33457eb17d99cee943d6e8-74b3115e391b453e","baggage":"sentry-trace_id=b2ba98fa6e33457eb17d99cee943d6e8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72NB81X6DX9ZB8PWC9JYDR7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.982Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2f17a73233ca8a0b3eb2afcb: start","job_status":"start","queue_duration_s":0.791341,"scheduling_latency_s":0.791356} gitlab # [ 268.744914] gitaly[919]: time="2026-11-10T12:01:26.586Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M72NB89S33MT5CWP07430JJW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T12:01:56.439" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:26.440" pid=919 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-11-10 12:00:19.645698942 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root gitlab # [ 268.745509] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.870Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"2f17a73233ca8a0b3eb2afcb","created_at":"2026-11-10T12:01:25.967Z","trace_propagation_headers":{"sentry-trace":"b2ba98fa6e33457eb17d99cee943d6e8-74b3115e391b453e","baggage":"sentry-trace_id=b2ba98fa6e33457eb17d99cee943d6e8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72NB81X6DX9ZB8PWC9JYDR7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:25.982Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2f17a73233ca8a0b3eb2afcb: done: 0.09609 sec","job_status":"done","queue_duration_s":0.791341,"scheduling_latency_s":0.791356,"gitaly_calls":2,"gitaly_duration_s":0.006426,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.006121000000000001,"redis_read_bytes":80,"redis_write_bytes":5119,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000619,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000876,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000202,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1130,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001056,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000154,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000313,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002282,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065432,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.067223348,"duration_s":0.09609,"completed_at":"2026-11-10T12:01:26.870Z","load_balancing_strategy":"primary","db_duration_s":0.022255,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 268.790580] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.871Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c2ff00d5f2f38f592a6f2037","created_at":"2026-11-10T12:01:26.434Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.436Z","job_size_bytes":23,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c2ff00d5f2f38f592a6f2037: start","job_status":"start","queue_duration_s":0.435324,"scheduling_latency_s":0.435339} gitlab # [ 268.830736] gitaly[919]: time="2026-11-10T12:01:26.587Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=12 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB8YZ6GYHD7SH1V0P0EKN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:26.565" grpc.time_ms=21.911 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 268.831530] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.899Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c2ff00d5f2f38f592a6f2037","created_at":"2026-11-10T12:01:26.434Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.436Z","job_size_bytes":23,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c2ff00d5f2f38f592a6f2037: done: 0.027673 sec","job_status":"done","queue_duration_s":0.435324,"scheduling_latency_s":0.435339,"redis_calls":7,"redis_duration_s":0.001132,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000267,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027673,"completed_at":"2026-11-10T12:01:26.899Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005589,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 268.844903] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.900Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"4eed8d3a14ac20b33290c8cf","created_at":"2026-11-10T12:01:26.701Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.703Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-4eed8d3a14ac20b33290c8cf: start","job_status":"start","queue_duration_s":0.197141,"scheduling_latency_s":0.197157} gitlab # [ 268.860132] gitaly[919]: time="2026-11-10T12:01:26.613Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB90N5BAKK3M7BAEX8W1G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:56.512" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:26.613" grpc.time_ms=0.534 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 268.899648] gitaly[919]: time="2026-11-10T12:01:26.642Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72NB922NJ1PS5PD3TDBZADH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:26.635" pid=919 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 268.900463] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.916Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"4eed8d3a14ac20b33290c8cf","created_at":"2026-11-10T12:01:26.701Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.703Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-4eed8d3a14ac20b33290c8cf: done: 0.016509 sec","job_status":"done","queue_duration_s":0.197141,"scheduling_latency_s":0.197157,"redis_calls":7,"redis_duration_s":0.000975,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000717,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000258,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011143,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016509,"completed_at":"2026-11-10T12:01:26.916Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00262,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.944745] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.917Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b917cfd084ccccf231d7a81c","created_at":"2026-11-10T12:01:26.710Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.713Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b917cfd084ccccf231d7a81c: start","job_status":"start","queue_duration_s":0.204231,"scheduling_latency_s":0.204246} gitlab # [ 268.945677] gitaly[919]: time="2026-11-10T12:01:26.642Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB922NJ1PS5PD3TDBZADH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:26.635" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 268.946460] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.935Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b917cfd084ccccf231d7a81c","created_at":"2026-11-10T12:01:26.710Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.713Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b917cfd084ccccf231d7a81c: done: 0.01748 sec","job_status":"done","queue_duration_s":0.204231,"scheduling_latency_s":0.204246,"redis_calls":7,"redis_duration_s":0.001067,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000797,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01748,"completed_at":"2026-11-10T12:01:26.935Z","load_balancing_strategy":"primary","db_duration_s":0.002741,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.986543] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.936Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"47c606a325810fd534d6d06e","created_at":"2026-11-10T12:01:26.720Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.722Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-47c606a325810fd534d6d06e: start","job_status":"start","queue_duration_s":0.213849,"scheduling_latency_s":0.213863} gitlab # [ 268.995485] gitaly[919]: time="2026-11-10T12:01:26.643Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB922NJ1PS5PD3TDBZADH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:26.635" grpc.time_ms=7.338 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 269.008259] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.955Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"47c606a325810fd534d6d06e","created_at":"2026-11-10T12:01:26.720Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.722Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-47c606a325810fd534d6d06e: done: 0.019225 sec","job_status":"done","queue_duration_s":0.213849,"scheduling_latency_s":0.213863,"redis_calls":10,"redis_duration_s":0.001356,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000416,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013284,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019225,"completed_at":"2026-11-10T12:01:26.955Z","load_balancing_strategy":"primary","db_duration_s":0.002901,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 269.068551] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.956Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"f410c91584095f4cf0cc3227","created_at":"2026-11-10T12:01:26.722Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.723Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-f410c91584095f4cf0cc3227: start","job_status":"start","queue_duration_s":0.23269,"scheduling_latency_s":0.232705} gitlab # [ 269.069256] gitaly[919]: time="2026-11-10T12:01:26.687Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M72NB89S33MT5CWP07430JJW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T12:01:56.439" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:26.440" pid=919 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root gitlab # [ 269.074589] gitaly[919]: time="2026-11-10T12:01:26.688Z" level=info msg="optimized repository" clean_stale_data_time=93.998539ms clean_work_trees_time="101.41µs" commit_graph_time="3.911µs" correlation_id=01M72NB89S33MT5CWP07430JJW gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="4.191µs" pid=919 prune_time="3.352µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="4.469µs" storage_name=default system=housekeeping total_time=94.207504ms gitlab # [ 269.079970] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.973Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"f410c91584095f4cf0cc3227","created_at":"2026-11-10T12:01:26.722Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.723Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-f410c91584095f4cf0cc3227: done: 0.016796 sec","job_status":"done","queue_duration_s":0.23269,"scheduling_latency_s":0.232705,"redis_calls":7,"redis_duration_s":0.0010479999999999999,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000332,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011359,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016796,"completed_at":"2026-11-10T12:01:26.973Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002489,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 269.142874] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.974Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6c324e91d79fdb7fa085ef38","created_at":"2026-11-10T12:01:26.724Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.725Z","job_size_bytes":19,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6c324e91d79fdb7fa085ef38: start","job_status":"start","queue_duration_s":0.249195,"scheduling_latency_s":0.24921} gitlab # [ 269.143752] gitaly[919]: time="2026-11-10T12:01:26.688Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M72NB89S33MT5CWP07430JJW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T12:01:56.439" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:26.440" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.144409] gitaly[919]: time="2026-11-10T12:01:26.696Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72NB89S33MT5CWP07430JJW diskcache=9a2d1a4b-ffcd-4aec-96b4-5bb1979250ed grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T12:01:56.439" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:26.440" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.182357] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.989Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6c324e91d79fdb7fa085ef38","created_at":"2026-11-10T12:01:26.724Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.725Z","job_size_bytes":19,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6c324e91d79fdb7fa085ef38: done: 0.015353 sec","job_status":"done","queue_duration_s":0.249195,"scheduling_latency_s":0.24921,"redis_calls":8,"redis_duration_s":0.001128,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000257,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009703,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015353,"completed_at":"2026-11-10T12:01:26.989Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002211,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 269.187482] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:26.990Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"462ad65127fb0917894303eb","created_at":"2026-11-10T12:01:26.726Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.727Z","job_size_bytes":66,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-462ad65127fb0917894303eb: start","job_status":"start","queue_duration_s":0.263742,"scheduling_latency_s":0.263757} gitlab # [ 269.230577] gitaly[919]: time="2026-11-10T12:01:26.696Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=74 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=8455 command.oublock=0 command.real_time_ms=232 command.spawn_token_fork_ms=91 command.system_time_ms=43 command.user_time_ms=31 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB89S33MT5CWP07430JJW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T12:01:56.439" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:26.440" grpc.time_ms=256.529 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.231166] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:27.004Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"462ad65127fb0917894303eb","created_at":"2026-11-10T12:01:26.726Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.727Z","job_size_bytes":66,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-462ad65127fb0917894303eb: done: 0.013546 sec","job_status":"done","queue_duration_s":0.263742,"scheduling_latency_s":0.263757,"redis_calls":7,"redis_duration_s":0.000936,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000682,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009222,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013546,"completed_at":"2026-11-10T12:01:27.004Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001594,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 269.253532] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:27.005Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"88167a17143bac0cdc5db7fd","created_at":"2026-11-10T12:01:26.727Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.728Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-88167a17143bac0cdc5db7fd: start","job_status":"start","queue_duration_s":0.276934,"scheduling_latency_s":0.276948} gitlab # [ 269.306059] gitaly[919]: time="2026-11-10T12:01:26.741Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB89S33MT5CWP07430JJW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T12:01:56.738" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:26.739" grpc.time_ms=1.371 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.306784] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:27.026Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"88167a17143bac0cdc5db7fd","created_at":"2026-11-10T12:01:26.727Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.728Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-88167a17143bac0cdc5db7fd: done: 0.021066 sec","job_status":"done","queue_duration_s":0.276934,"scheduling_latency_s":0.276948,"redis_calls":7,"redis_duration_s":0.001335,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000425,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021066,"completed_at":"2026-11-10T12:01:27.026Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003917,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 269.316735] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:27.027Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"abc0ef219f721eb9f616df0c","created_at":"2026-11-10T12:01:26.728Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.729Z","job_size_bytes":6,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-abc0ef219f721eb9f616df0c: start","job_status":"start","queue_duration_s":0.297673,"scheduling_latency_s":0.297689} gitlab # [ 269.330833] gitaly[919]: time="2026-11-10T12:01:26.761Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB89S33MT5CWP07430JJW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:56.760" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:26.761" grpc.time_ms=0.449 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.376705] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:27.037Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"abc0ef219f721eb9f616df0c","created_at":"2026-11-10T12:01:26.728Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.729Z","job_size_bytes":6,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-abc0ef219f721eb9f616df0c: done: 0.009676 sec","job_status":"done","queue_duration_s":0.297673,"scheduling_latency_s":0.297689,"redis_calls":7,"redis_duration_s":0.001142,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009676,"completed_at":"2026-11-10T12:01:27.037Z","load_balancing_strategy":"primary","db_duration_s":0.000293,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 269.378847] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:27.038Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"650cfbadd2d8f8f7103213a3","created_at":"2026-11-10T12:01:26.730Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.731Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-650cfbadd2d8f8f7103213a3: start","job_status":"start","queue_duration_s":0.30726,"scheduling_latency_s":0.307273} gitlab # [ 269.425601] gitaly[919]: time="2026-11-10T12:01:26.830Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NB81X6DX9ZB8PWC9JYDR7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T12:01:56.829" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:26.829" grpc.time_ms=1.063 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.426429] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:27.052Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"650cfbadd2d8f8f7103213a3","created_at":"2026-11-10T12:01:26.730Z","trace_propagation_headers":{"sentry-trace":"922e1f103d3d4855a4f67e70c09b451a-9f3a204349a14328","baggage":"sentry-trace_id=922e1f103d3d4855a4f67e70c09b451a,sentry-release=Unknown"},"correlation_id":"01M72NB89S33MT5CWP07430JJW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.731Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-650cfbadd2d8f8f7103213a3: done: 0.014452 sec","job_status":"done","queue_duration_s":0.30726,"scheduling_latency_s":0.307273,"redis_calls":7,"redis_duration_s":0.001058,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000802,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014452,"completed_at":"2026-11-10T12:01:27.052Z","load_balancing_strategy":"primary","db_duration_s":0.002105,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 269.472825] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:27.053Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"7408ce067671c84a7cda6431","created_at":"2026-11-10T12:01:26.818Z","trace_propagation_headers":{"sentry-trace":"b2ba98fa6e33457eb17d99cee943d6e8-41cbce1730d6486b","baggage":"sentry-trace_id=b2ba98fa6e33457eb17d99cee943d6e8,sentry-release=Unknown"},"correlation_id":"01M72NB81X6DX9ZB8PWC9JYDR7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.820Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7408ce067671c84a7cda6431: start","job_status":"start","queue_duration_s":0.233665,"scheduling_latency_s":0.23368} gitlab # [ 269.473754] gitaly[919]: time="2026-11-10T12:01:26.860Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NB81X6DX9ZB8PWC9JYDR7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:56.859" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:26.859" grpc.time_ms=0.476 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.474513] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:27.074Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"7408ce067671c84a7cda6431","created_at":"2026-11-10T12:01:26.818Z","trace_propagation_headers":{"sentry-trace":"b2ba98fa6e33457eb17d99cee943d6e8-41cbce1730d6486b","baggage":"sentry-trace_id=b2ba98fa6e33457eb17d99cee943d6e8,sentry-release=Unknown"},"correlation_id":"01M72NB81X6DX9ZB8PWC9JYDR7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:26.820Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7408ce067671c84a7cda6431: done: 0.020972 sec","job_status":"done","queue_duration_s":0.233665,"scheduling_latency_s":0.23368,"redis_calls":7,"redis_duration_s":0.001188,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020972,"completed_at":"2026-11-10T12:01:27.074Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002967,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses gitlab # [ 269.614356] gitaly[919]: time="2026-11-10T12:01:27.746Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NBA52X3YEDMW3T9N02XZ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:57.645" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:27.745" grpc.time_ms=0.611 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 269.661687] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:27 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 78 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses, in 0.12 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses gitlab # [ 270.757864] gitaly[919]: time="2026-11-10T12:01:28.889Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NBB80W8FFG9NJAE94JKXY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:01:58.787" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:28.887" grpc.time_ms=1.024 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 270.803230] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:28 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 101 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses, in 0.14 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses gitlab # [ 271.902085] gitaly[919]: time="2026-11-10T12:01:30.033Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NBCBQATE6QBE2XVBFR2EZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:02:00.031" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:30.032" grpc.time_ms=0.942 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 271.965785] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 121 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses, in 0.17 seconds) gitlab # [ 272.010770] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.141Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"0af016a29ca17a0d7fc34a09","created_at":"2026-11-10T12:01:30.132Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80022a99a17650694e16f143b74ebaa2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T12:01:30.139Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-0af016a29ca17a0d7fc34a09: start","job_status":"start","queue_duration_s":0.002193,"scheduling_latency_s":0.002244} gitlab # [ 272.125664] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.257Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"0af016a29ca17a0d7fc34a09","created_at":"2026-11-10T12:01:30.132Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"80022a99a17650694e16f143b74ebaa2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T12:01:30.139Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-0af016a29ca17a0d7fc34a09: done: 0.115608 sec","job_status":"done","queue_duration_s":0.002193,"scheduling_latency_s":0.002244,"redis_calls":9,"redis_duration_s":0.01445,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00219,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003206,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.009054,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031741,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.115608,"completed_at":"2026-11-10T12:01:30.257Z","load_balancing_strategy":"primary","db_duration_s":0.0136,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 272.200975] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.332Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"fddae5620f79938e85d96ea2","created_at":"2026-11-10T12:01:30.181Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.204Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-fddae5620f79938e85d96ea2: start","job_status":"start","queue_duration_s":0.127792,"scheduling_latency_s":0.127844} gitlab # [ 272.526438] nix-daemon[2905]: accepted connection from pid 3845, user nixbot gitlab # [ 272.542572] nix-daemon[2905]: accepted connection from pid 3749, user nixbot gitlab # [ 272.644792] gitaly[919]: time="2026-11-10T12:01:30.776Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NBD2Y5VG3MVZJSDBY97BW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:02:00.675" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:30.776" grpc.time_ms=0.741 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 272.659876] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.791Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"fddae5620f79938e85d96ea2","created_at":"2026-11-10T12:01:30.181Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.204Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-fddae5620f79938e85d96ea2: done: 0.458711 sec","job_status":"done","queue_duration_s":0.127792,"scheduling_latency_s":0.127844,"redis_calls":70,"redis_duration_s":0.19384900000000002,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.010746,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001502,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.176772,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004829,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.136585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.458711,"completed_at":"2026-11-10T12:01:30.790Z","load_balancing_strategy":"primary","db_duration_s":0.031731,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 272.667814] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"4bdebbd2621e7c98030ad3a4","created_at":"2026-11-10T12:01:30.223Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"336b38a763d05f28bb31cb573ba2a73f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.329Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-4bdebbd2621e7c98030ad3a4: start","job_status":"start","queue_duration_s":0.470458,"scheduling_latency_s":0.470498} gitlab # [ 272.701240] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.833Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 272.710773] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.842Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"4bdebbd2621e7c98030ad3a4","created_at":"2026-11-10T12:01:30.223Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"336b38a763d05f28bb31cb573ba2a73f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.329Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-4bdebbd2621e7c98030ad3a4: done: 0.043103 sec","job_status":"done","queue_duration_s":0.470458,"scheduling_latency_s":0.470498,"redis_calls":9,"redis_duration_s":0.005144,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00069,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002443,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002011,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012784,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043103,"completed_at":"2026-11-10T12:01:30.842Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009802,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 272.715513] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.844Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"1526b18c5067a303aec3bf28","created_at":"2026-11-10T12:01:30.352Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34dc3cb4e66573481ab546ee33706bfb","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.368Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-1526b18c5067a303aec3bf28: start","job_status":"start","queue_duration_s":0.47573,"scheduling_latency_s":0.475748} gitlab # [ 272.760920] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.892Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"1526b18c5067a303aec3bf28","created_at":"2026-11-10T12:01:30.352Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34dc3cb4e66573481ab546ee33706bfb","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.368Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-1526b18c5067a303aec3bf28: done: 0.0482 sec","job_status":"done","queue_duration_s":0.47573,"scheduling_latency_s":0.475748,"redis_calls":9,"redis_duration_s":0.002672,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001261,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001147,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000264,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0482,"completed_at":"2026-11-10T12:01:30.892Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016939,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 272.766859] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.893Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"7e0c6dba97efeb1889f586ad","created_at":"2026-11-10T12:01:30.406Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0710c9712329a3cbed6f05749ceb2671","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.424Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-7e0c6dba97efeb1889f586ad: start","job_status":"start","queue_duration_s":0.469303,"scheduling_latency_s":0.469319} gitlab # [ 272.785877] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.917Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"7e0c6dba97efeb1889f586ad","created_at":"2026-11-10T12:01:30.406Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0710c9712329a3cbed6f05749ceb2671","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.424Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-7e0c6dba97efeb1889f586ad: done: 0.02388 sec","job_status":"done","queue_duration_s":0.469303,"scheduling_latency_s":0.469319,"redis_calls":18,"redis_duration_s":0.002324,"redis_read_bytes":20,"redis_write_bytes":4694,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000565,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000194,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001193,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000372,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02388,"completed_at":"2026-11-10T12:01:30.917Z","load_balancing_strategy":"primary","db_duration_s":0.002136,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 272.817290] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.918Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"44bd24fca3367ab1b0a09fb8","created_at":"2026-11-10T12:01:30.437Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c8a9788d7a10a3f530df399a4f603bf2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.456Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-44bd24fca3367ab1b0a09fb8: start","job_status":"start","queue_duration_s":0.461856,"scheduling_latency_s":0.461872} gitlab # [ 272.861938] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.963Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"44bd24fca3367ab1b0a09fb8","created_at":"2026-11-10T12:01:30.437Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c8a9788d7a10a3f530df399a4f603bf2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.456Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-44bd24fca3367ab1b0a09fb8: done: 0.044481 sec","job_status":"done","queue_duration_s":0.461856,"scheduling_latency_s":0.461872,"redis_calls":13,"redis_duration_s":0.0018340000000000001,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000491,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000639,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000355073,"exclusive_lock_hold_duration_s":0.029132473,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.044481,"completed_at":"2026-11-10T12:01:30.963Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014459,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 272.870727] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.964Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"8aca83dc3fa557d195610bc0","created_at":"2026-11-10T12:01:30.467Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21d882796d6db7a6c6907ccb70d4e1b2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.480Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-8aca83dc3fa557d195610bc0: start","job_status":"start","queue_duration_s":0.483867,"scheduling_latency_s":0.483883} gitlab # [ 272.917338] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.985Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"8aca83dc3fa557d195610bc0","created_at":"2026-11-10T12:01:30.467Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21d882796d6db7a6c6907ccb70d4e1b2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.480Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-8aca83dc3fa557d195610bc0: done: 0.021582 sec","job_status":"done","queue_duration_s":0.483867,"scheduling_latency_s":0.483883,"redis_calls":11,"redis_duration_s":0.001709,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000669,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000794,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010214,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000487772,"exclusive_lock_hold_duration_s":0.00794682,"duration_s":0.021582,"completed_at":"2026-11-10T12:01:30.985Z","load_balancing_strategy":"primary","db_duration_s":0.005962,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 272.920608] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:30.986Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3f372966f5ed5343e42bca5f","created_at":"2026-11-10T12:01:30.497Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40fa1efaa0c94ffb98a9d56262e5117d","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.510Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3f372966f5ed5343e42bca5f: start","job_status":"start","queue_duration_s":0.476246,"scheduling_latency_s":0.47626} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses gitlab # [ 272.985280] gitaly[919]: time="2026-11-10T12:01:31.116Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72NBD2Y5VG3MVZJSDBY97BW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:01:40.385" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:31.105" grpc.time_ms=10.575 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 273.039060] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6279a8528aa5cc381d8ddfcd046a8d644342b5aa HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 451 gitlab # [ 273.043480] gitaly[919]: time="2026-11-10T12:01:31.175Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NBDG4MYTKTH0DHEG5CF5X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:02:01.074" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:31.174" grpc.time_ms=0.51 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 273.061586] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses HTTP/1.1" 200 677 "" "curl/8.21.0" 52 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses, in 0.09 seconds) gitlab # [ 273.097953] gitaly[919]: time="2026-11-10T12:01:31.230Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NBDHMYFJ04EB36C5ZA9EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:02:01.129" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:31.229" grpc.time_ms=0.776 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 273.140075] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.271Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"3f372966f5ed5343e42bca5f","created_at":"2026-11-10T12:01:30.497Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40fa1efaa0c94ffb98a9d56262e5117d","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.510Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-3f372966f5ed5343e42bca5f: done: 0.284493 sec","job_status":"done","queue_duration_s":0.476246,"scheduling_latency_s":0.47626,"redis_calls":564,"redis_duration_s":0.08689100000000001,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000285,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000817,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.085789,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.212505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.284493,"completed_at":"2026-11-10T12:01:31.271Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.140948] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.272Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"cc91d82c2e5f37958032f395","created_at":"2026-11-10T12:01:30.529Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c710e812972b21d810edf5431d54c82e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.540Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-cc91d82c2e5f37958032f395: start","job_status":"start","queue_duration_s":0.731948,"scheduling_latency_s":0.731971} gitlab # [ 273.162607] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.294Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"cc91d82c2e5f37958032f395","created_at":"2026-11-10T12:01:30.529Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c710e812972b21d810edf5431d54c82e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.540Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-cc91d82c2e5f37958032f395: done: 0.021678 sec","job_status":"done","queue_duration_s":0.731948,"scheduling_latency_s":0.731971,"redis_calls":9,"redis_duration_s":0.0014320000000000001,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000359,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000808,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01398,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021678,"completed_at":"2026-11-10T12:01:31.294Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002468,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.216879] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.295Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8f3da44aaa17fb541503b664","created_at":"2026-11-10T12:01:30.556Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"864e59af3f9639e5c511ec44975d185e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.568Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8f3da44aaa17fb541503b664: start","job_status":"start","queue_duration_s":0.727115,"scheduling_latency_s":0.727136} gitlab # [ 273.217782] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.313Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8f3da44aaa17fb541503b664","created_at":"2026-11-10T12:01:30.556Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"864e59af3f9639e5c511ec44975d185e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.568Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8f3da44aaa17fb541503b664: done: 0.018086 sec","job_status":"done","queue_duration_s":0.727115,"scheduling_latency_s":0.727136,"redis_calls":11,"redis_duration_s":0.001513,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00057,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018086,"completed_at":"2026-11-10T12:01:31.313Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003126,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.260195] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.314Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"f3d5f25d17f4fbb9581ffd2e","created_at":"2026-11-10T12:01:30.591Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"015282bdfe950baa6737c078c2bfebba","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.606Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-f3d5f25d17f4fbb9581ffd2e: start","job_status":"start","queue_duration_s":0.708059,"scheduling_latency_s":0.708074} gitlab # [ 273.260872] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.334Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"f3d5f25d17f4fbb9581ffd2e","created_at":"2026-11-10T12:01:30.591Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"015282bdfe950baa6737c078c2bfebba","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.606Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-f3d5f25d17f4fbb9581ffd2e: done: 0.019312 sec","job_status":"done","queue_duration_s":0.708059,"scheduling_latency_s":0.708074,"redis_calls":9,"redis_duration_s":0.00126,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000252,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00074,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.019312,"completed_at":"2026-11-10T12:01:31.334Z","load_balancing_strategy":"primary","db_duration_s":0.005072,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.317702] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.335Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"9592cad8e7f107580bc1720a","created_at":"2026-11-10T12:01:30.620Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e32e94b23d24387f2472e3ec01fdb6ea","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.634Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-9592cad8e7f107580bc1720a: start","job_status":"start","queue_duration_s":0.700926,"scheduling_latency_s":0.700942} gitlab # [ 273.318402] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.348Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"9592cad8e7f107580bc1720a","created_at":"2026-11-10T12:01:30.620Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e32e94b23d24387f2472e3ec01fdb6ea","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.634Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-9592cad8e7f107580bc1720a: done: 0.013658 sec","job_status":"done","queue_duration_s":0.700926,"scheduling_latency_s":0.700942,"redis_calls":11,"redis_duration_s":0.0016380000000000001,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000733,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000341664,"exclusive_lock_hold_duration_s":0.001616965,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.013658,"completed_at":"2026-11-10T12:01:31.348Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000934,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.371719] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.349Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7b6b8a08c74419aa6706134c","created_at":"2026-11-10T12:01:30.648Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4ea3937a63e6e7b0031dfcea49d44e3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.665Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7b6b8a08c74419aa6706134c: start","job_status":"start","queue_duration_s":0.683849,"scheduling_latency_s":0.683862} gitlab # [ 273.372516] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.365Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7b6b8a08c74419aa6706134c","created_at":"2026-11-10T12:01:30.648Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4ea3937a63e6e7b0031dfcea49d44e3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.665Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7b6b8a08c74419aa6706134c: done: 0.016356 sec","job_status":"done","queue_duration_s":0.683849,"scheduling_latency_s":0.683862,"redis_calls":9,"redis_duration_s":0.001576,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000285,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000369,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016356,"completed_at":"2026-11-10T12:01:31.365Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000591,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.421826] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9639f76e3e234d69eecf3716","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9639f76e3e234d69eecf3716: start","job_status":"start","queue_duration_s":0.609125,"scheduling_latency_s":0.609147} gitlab # [ 273.422722] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9639f76e3e234d69eecf3716","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9639f76e3e234d69eecf3716: done: 0.014357 sec","job_status":"done","queue_duration_s":0.609125,"scheduling_latency_s":0.609147,"redis_calls":15,"redis_duration_s":0.001882,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000252,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001138,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014357,"completed_at":"2026-11-10T12:01:31.381Z","load_balancing_strategy":"primary","db_duration_s":0.000285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.471295] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eea253cc0097901f6ce9685e","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eea253cc0097901f6ce9685e: start","job_status":"start","queue_duration_s":0.624588,"scheduling_latency_s":0.624603} gitlab # [ 273.472444] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eea253cc0097901f6ce9685e","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eea253cc0097901f6ce9685e: done: 0.009679 sec","job_status":"done","queue_duration_s":0.624588,"scheduling_latency_s":0.624603,"redis_calls":13,"redis_duration_s":0.001488,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009679,"completed_at":"2026-11-10T12:01:31.392Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.512146] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08488559191f1dc0c8ccc7bf","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08488559191f1dc0c8ccc7bf: start","job_status":"start","queue_duration_s":0.635209,"scheduling_latency_s":0.635223} gitlab # [ 273.512992] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08488559191f1dc0c8ccc7bf","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08488559191f1dc0c8ccc7bf: done: 0.009655 sec","job_status":"done","queue_duration_s":0.635209,"scheduling_latency_s":0.635223,"redis_calls":13,"redis_duration_s":0.001657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009655,"completed_at":"2026-11-10T12:01:31.402Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.531182] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9bc00b370741f537d0d57133","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9bc00b370741f537d0d57133: start","job_status":"start","queue_duration_s":0.645736,"scheduling_latency_s":0.645749} gitlab # [ 273.574297] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9bc00b370741f537d0d57133","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9bc00b370741f537d0d57133: done: 0.041979 sec","job_status":"done","queue_duration_s":0.645736,"scheduling_latency_s":0.645749,"redis_calls":13,"redis_duration_s":0.0016979999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041979,"completed_at":"2026-11-10T12:01:31.445Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.577728] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.446Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"083d467966afc07865d0a985","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-083d467966afc07865d0a985: start","job_status":"start","queue_duration_s":0.688671,"scheduling_latency_s":0.688685} gitlab # [ 273.589576] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.458Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"083d467966afc07865d0a985","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-083d467966afc07865d0a985: done: 0.011925 sec","job_status":"done","queue_duration_s":0.688671,"scheduling_latency_s":0.688685,"redis_calls":13,"redis_duration_s":0.002351,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001521,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011925,"completed_at":"2026-11-10T12:01:31.458Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.638051] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.459Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6515e94ab1e311eea283269d","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6515e94ab1e311eea283269d: start","job_status":"start","queue_duration_s":0.701632,"scheduling_latency_s":0.701645} gitlab # [ 273.638774] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6515e94ab1e311eea283269d","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6515e94ab1e311eea283269d: done: 0.009618 sec","job_status":"done","queue_duration_s":0.701632,"scheduling_latency_s":0.701645,"redis_calls":13,"redis_duration_s":0.001617,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009618,"completed_at":"2026-11-10T12:01:31.469Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.680090] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a99fc5633678ee4a6944015","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a99fc5633678ee4a6944015: start","job_status":"start","queue_duration_s":0.712116,"scheduling_latency_s":0.71213} gitlab # [ 273.680849] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a99fc5633678ee4a6944015","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a99fc5633678ee4a6944015: done: 0.009895 sec","job_status":"done","queue_duration_s":0.712116,"scheduling_latency_s":0.71213,"redis_calls":13,"redis_duration_s":0.001696,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001164,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009895,"completed_at":"2026-11-10T12:01:31.479Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.729718] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15989cd7c8264ba844433807","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15989cd7c8264ba844433807: start","job_status":"start","queue_duration_s":0.722866,"scheduling_latency_s":0.722879} gitlab # [ 273.730554] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15989cd7c8264ba844433807","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15989cd7c8264ba844433807: done: 0.009395 sec","job_status":"done","queue_duration_s":0.722866,"scheduling_latency_s":0.722879,"redis_calls":13,"redis_duration_s":0.001505,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009395,"completed_at":"2026-11-10T12:01:31.490Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.762954] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e1774c6a6869ba953d226e9","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e1774c6a6869ba953d226e9: start","job_status":"start","queue_duration_s":0.73311,"scheduling_latency_s":0.733123} gitlab # [ 273.774520] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.500Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e1774c6a6869ba953d226e9","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e1774c6a6869ba953d226e9: done: 0.009219 sec","job_status":"done","queue_duration_s":0.73311,"scheduling_latency_s":0.733123,"redis_calls":13,"redis_duration_s":0.0014689999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000486,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009219,"completed_at":"2026-11-10T12:01:31.500Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.823114] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ebf01208236945cf61f5493a","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ebf01208236945cf61f5493a: start","job_status":"start","queue_duration_s":0.743619,"scheduling_latency_s":0.743633} gitlab # [ 273.823896] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.512Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ebf01208236945cf61f5493a","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ebf01208236945cf61f5493a: done: 0.010569 sec","job_status":"done","queue_duration_s":0.743619,"scheduling_latency_s":0.743633,"redis_calls":13,"redis_duration_s":0.002678,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000849,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001829,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00628,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010569,"completed_at":"2026-11-10T12:01:31.512Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.880891] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4106fb36d8ee934799213d61","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4106fb36d8ee934799213d61: start","job_status":"start","queue_duration_s":0.755269,"scheduling_latency_s":0.755284} gitlab # [ 273.881838] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4106fb36d8ee934799213d61","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4106fb36d8ee934799213d61: done: 0.010587 sec","job_status":"done","queue_duration_s":0.755269,"scheduling_latency_s":0.755284,"redis_calls":13,"redis_duration_s":0.002659,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001788,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010587,"completed_at":"2026-11-10T12:01:31.523Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.927876] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14eb6d62d46a542d815b9430","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14eb6d62d46a542d815b9430: start","job_status":"start","queue_duration_s":0.766927,"scheduling_latency_s":0.767095} gitlab # [ 273.928772] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14eb6d62d46a542d815b9430","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14eb6d62d46a542d815b9430: done: 0.011111 sec","job_status":"done","queue_duration_s":0.766927,"scheduling_latency_s":0.767095,"redis_calls":13,"redis_duration_s":0.0030299999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001348,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001682,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011111,"completed_at":"2026-11-10T12:01:31.535Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.976769] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3fcf54ac93ef35e2a15b438","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3fcf54ac93ef35e2a15b438: start","job_status":"start","queue_duration_s":0.779154,"scheduling_latency_s":0.779168} gitlab # [ 273.977702] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3fcf54ac93ef35e2a15b438","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3fcf54ac93ef35e2a15b438: done: 0.012871 sec","job_status":"done","queue_duration_s":0.779154,"scheduling_latency_s":0.779168,"redis_calls":13,"redis_duration_s":0.0025960000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00181,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006359,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012871,"completed_at":"2026-11-10T12:01:31.549Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 273.994048] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52d12cb95c1c653b4485ac29","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52d12cb95c1c653b4485ac29: start","job_status":"start","queue_duration_s":0.79329,"scheduling_latency_s":0.793304} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses gitlab # [ 274.005150] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52d12cb95c1c653b4485ac29","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52d12cb95c1c653b4485ac29: done: 0.011047 sec","job_status":"done","queue_duration_s":0.79329,"scheduling_latency_s":0.793304,"redis_calls":13,"redis_duration_s":0.0026899999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001785,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011047,"completed_at":"2026-11-10T12:01:31.562Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.045133] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35ad7a8347fa9d779932d220","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35ad7a8347fa9d779932d220: start","job_status":"start","queue_duration_s":0.805485,"scheduling_latency_s":0.805498} gitlab # [ 274.051838] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35ad7a8347fa9d779932d220","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35ad7a8347fa9d779932d220: done: 0.010512 sec","job_status":"done","queue_duration_s":0.805485,"scheduling_latency_s":0.805498,"redis_calls":13,"redis_duration_s":0.002548,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000773,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001775,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010512,"completed_at":"2026-11-10T12:01:31.573Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.094648] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2018292dbb5acaaaeacbe03","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2018292dbb5acaaaeacbe03: start","job_status":"start","queue_duration_s":0.817224,"scheduling_latency_s":0.817238} gitlab # [ 274.095994] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2018292dbb5acaaaeacbe03","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2018292dbb5acaaaeacbe03: done: 0.013608 sec","job_status":"done","queue_duration_s":0.817224,"scheduling_latency_s":0.817238,"redis_calls":13,"redis_duration_s":0.002354,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001594,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013608,"completed_at":"2026-11-10T12:01:31.588Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.145502] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.589Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7d18402bf708768187da436","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7d18402bf708768187da436: start","job_status":"start","queue_duration_s":0.831703,"scheduling_latency_s":0.831717} gitlab # [ 274.146244] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7d18402bf708768187da436","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7d18402bf708768187da436: done: 0.010494 sec","job_status":"done","queue_duration_s":0.831703,"scheduling_latency_s":0.831717,"redis_calls":13,"redis_duration_s":0.001796,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000479,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001317,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010494,"completed_at":"2026-11-10T12:01:31.600Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.199175] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07073412a761d22f61ed2c4d","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07073412a761d22f61ed2c4d: start","job_status":"start","queue_duration_s":0.843273,"scheduling_latency_s":0.843287} gitlab # [ 274.199841] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07073412a761d22f61ed2c4d","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07073412a761d22f61ed2c4d: done: 0.011282 sec","job_status":"done","queue_duration_s":0.843273,"scheduling_latency_s":0.843287,"redis_calls":13,"redis_duration_s":0.0021780000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001516,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011282,"completed_at":"2026-11-10T12:01:31.612Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.223111] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d74ac6dc11ef78fe0e36244f","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d74ac6dc11ef78fe0e36244f: start","job_status":"start","queue_duration_s":0.855556,"scheduling_latency_s":0.855571} gitlab # [ 274.234487] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d74ac6dc11ef78fe0e36244f","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d74ac6dc11ef78fe0e36244f: done: 0.011078 sec","job_status":"done","queue_duration_s":0.855556,"scheduling_latency_s":0.855571,"redis_calls":13,"redis_duration_s":0.002101,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000671,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00143,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011078,"completed_at":"2026-11-10T12:01:31.624Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.295552] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"035b4ec9c85363b2ecc68bd3","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-035b4ec9c85363b2ecc68bd3: start","job_status":"start","queue_duration_s":0.867606,"scheduling_latency_s":0.867619} gitlab # [ 274.296417] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"035b4ec9c85363b2ecc68bd3","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-035b4ec9c85363b2ecc68bd3: done: 0.0107 sec","job_status":"done","queue_duration_s":0.867606,"scheduling_latency_s":0.867619,"redis_calls":13,"redis_duration_s":0.00212,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000652,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001468,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0107,"completed_at":"2026-11-10T12:01:31.636Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.339573] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20dc93f10d5ebc3e2995e8f5","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20dc93f10d5ebc3e2995e8f5: start","job_status":"start","queue_duration_s":0.879764,"scheduling_latency_s":0.879778} gitlab # [ 274.340791] gitaly[919]: time="2026-11-10T12:01:32.411Z" level=info msg="finished unary call with code OK" catfile.duration_ms=60 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=60 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NBEKBHT8897J50C130Q5Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:02:02.249" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:32.350" grpc.time_ms=61.007 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses, in 0.32 seconds) gitlab # [ 274.341592] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20dc93f10d5ebc3e2995e8f5","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20dc93f10d5ebc3e2995e8f5: done: 0.011045 sec","job_status":"done","queue_duration_s":0.879764,"scheduling_latency_s":0.879778,"redis_calls":13,"redis_duration_s":0.002157,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000737,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00142,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006771,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011045,"completed_at":"2026-11-10T12:01:31.648Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.381228] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6762869a07ce60ecf909df7c","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6762869a07ce60ecf909df7c: start","job_status":"start","queue_duration_s":0.892045,"scheduling_latency_s":0.892059} gitlab # [ 274.383682] gitaly[919]: time="2026-11-10T12:01:32.414Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72NBDHMYFJ04EB36C5ZA9EN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:01:41.265" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:31.265" grpc.time_ms=1148.53 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 274.396230] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:32 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6279a8528aa5cc381d8ddfcd046a8d644342b5aa HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1284 gitlab # [ 274.398041] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses HTTP/1.1" 200 677 "" "curl/8.21.0" 240 gitlab # [ 274.400812] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.661Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6762869a07ce60ecf909df7c","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6762869a07ce60ecf909df7c: done: 0.011072 sec","job_status":"done","queue_duration_s":0.892045,"scheduling_latency_s":0.892059,"redis_calls":13,"redis_duration_s":0.0021330000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001471,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011072,"completed_at":"2026-11-10T12:01:31.661Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.442291] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e65f778192df70d97e4f87c","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e65f778192df70d97e4f87c: start","job_status":"start","queue_duration_s":0.904387,"scheduling_latency_s":0.904401} gitlab # [ 274.446683] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e65f778192df70d97e4f87c","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e65f778192df70d97e4f87c: done: 0.017253 sec","job_status":"done","queue_duration_s":0.904387,"scheduling_latency_s":0.904401,"redis_calls":13,"redis_duration_s":0.003789,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00154,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002249,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012294,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017253,"completed_at":"2026-11-10T12:01:31.679Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.486286] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"642d51aa2a1f331fe2df3276","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-642d51aa2a1f331fe2df3276: start","job_status":"start","queue_duration_s":0.923012,"scheduling_latency_s":0.923028} gitlab # [ 274.492275] gitaly[919]: time="2026-11-10T12:01:32.560Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NBEVC4ZWJWQ2PQ7M3RVDR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:02:02.459" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:32.559" grpc.time_ms=0.588 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 274.535619] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"642d51aa2a1f331fe2df3276","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-642d51aa2a1f331fe2df3276: done: 0.011472 sec","job_status":"done","queue_duration_s":0.923012,"scheduling_latency_s":0.923028,"redis_calls":13,"redis_duration_s":0.002571,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001745,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011472,"completed_at":"2026-11-10T12:01:31.692Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.576355] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"930f4d3aa7963ac96b9a8d70","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-930f4d3aa7963ac96b9a8d70: start","job_status":"start","queue_duration_s":0.935929,"scheduling_latency_s":0.935943} gitlab # [ 274.577202] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:32 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/6279a8528aa5cc381d8ddfcd046a8d644342b5aa HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 102 gitlab # [ 274.577781] gitaly[919]: time="2026-11-10T12:01:32.588Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72NBEVC4ZWJWQ2PQ7M3RVDR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T12:01:42.251" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T12:01:32.581" grpc.time_ms=6.59 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 274.578403] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"930f4d3aa7963ac96b9a8d70","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-930f4d3aa7963ac96b9a8d70: done: 0.011204 sec","job_status":"done","queue_duration_s":0.935929,"scheduling_latency_s":0.935943,"redis_calls":13,"redis_duration_s":0.002497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000813,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001684,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011204,"completed_at":"2026-11-10T12:01:31.705Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.598705] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.706Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce0cbc2cabc9bc98da1ea60a","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce0cbc2cabc9bc98da1ea60a: start","job_status":"start","queue_duration_s":0.948467,"scheduling_latency_s":0.948482} gitlab # [ 274.609796] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce0cbc2cabc9bc98da1ea60a","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce0cbc2cabc9bc98da1ea60a: done: 0.010668 sec","job_status":"done","queue_duration_s":0.948467,"scheduling_latency_s":0.948482,"redis_calls":13,"redis_duration_s":0.0016380000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010668,"completed_at":"2026-11-10T12:01:31.716Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.649096] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.717Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70d68ca4363fc82da9aa3cfa","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70d68ca4363fc82da9aa3cfa: start","job_status":"start","queue_duration_s":0.959745,"scheduling_latency_s":0.959759} gitlab # [ 274.655259] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.728Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70d68ca4363fc82da9aa3cfa","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70d68ca4363fc82da9aa3cfa: done: 0.010397 sec","job_status":"done","queue_duration_s":0.959745,"scheduling_latency_s":0.959759,"redis_calls":13,"redis_duration_s":0.0018050000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001293,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010397,"completed_at":"2026-11-10T12:01:31.728Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.714508] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.728Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f02eb8dda12ad9724f8e342","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f02eb8dda12ad9724f8e342: start","job_status":"start","queue_duration_s":0.970985,"scheduling_latency_s":0.970999} gitlab # [ 274.715183] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f02eb8dda12ad9724f8e342","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f02eb8dda12ad9724f8e342: done: 0.009608 sec","job_status":"done","queue_duration_s":0.970985,"scheduling_latency_s":0.970999,"redis_calls":13,"redis_duration_s":0.0018700000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000612,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001258,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009608,"completed_at":"2026-11-10T12:01:31.738Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.759285] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.739Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24ec6282477a6573e3f9aaf2","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24ec6282477a6573e3f9aaf2: start","job_status":"start","queue_duration_s":0.981448,"scheduling_latency_s":0.981462} gitlab # [ 274.760450] nixbot[2744]: {"ts": "2026-11-10T12:01:32.805Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 2, "sha": "6279a8528aa5cc381d8ddfcd046a8d644342b5aa", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []} gitlab # [ 274.761335] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24ec6282477a6573e3f9aaf2","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24ec6282477a6573e3f9aaf2: done: 0.009047 sec","job_status":"done","queue_duration_s":0.981448,"scheduling_latency_s":0.981462,"redis_calls":13,"redis_duration_s":0.00147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000475,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009047,"completed_at":"2026-11-10T12:01:31.748Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.801245] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d513e8e7b89a413627dcfbe","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d513e8e7b89a413627dcfbe: start","job_status":"start","queue_duration_s":0.991317,"scheduling_latency_s":0.99133} gitlab # [ 274.801932] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d513e8e7b89a413627dcfbe","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d513e8e7b89a413627dcfbe: done: 0.009106 sec","job_status":"done","queue_duration_s":0.991317,"scheduling_latency_s":0.99133,"redis_calls":13,"redis_duration_s":0.0014520000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000473,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009106,"completed_at":"2026-11-10T12:01:31.758Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.851101] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"871a5f6d4e1565cf28feeb39","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-871a5f6d4e1565cf28feeb39: start","job_status":"start","queue_duration_s":1.001251,"scheduling_latency_s":1.001265} gitlab # [ 274.851761] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:32 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 39 gitlab # [ 274.852198] gitlab-workhorse[825]: time="2026-11-10T12:01:32Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 274.852665] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:32 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 50 gitlab # [ 274.853194] gitaly[919]: time="2026-11-10T12:01:32.899Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72NBF5WQQNWS9997HCQHVV4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:32.892" grpc.time_ms=6.706 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 274.858980] gitaly[919]: time="2026-11-10T12:01:32.950Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72NBF7498F8J234B290074C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:32.943" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 274.899920] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"871a5f6d4e1565cf28feeb39","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-871a5f6d4e1565cf28feeb39: done: 0.009247 sec","job_status":"done","queue_duration_s":1.001251,"scheduling_latency_s":1.001265,"redis_calls":13,"redis_duration_s":0.0014939999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00048,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009247,"completed_at":"2026-11-10T12:01:31.768Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.902830] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f0698f660ab09090b37bde2","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f0698f660ab09090b37bde2: start","job_status":"start","queue_duration_s":1.011305,"scheduling_latency_s":1.011319} gitlab # [ 274.947849] gitaly[919]: time="2026-11-10T12:01:32.950Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NBF7498F8J234B290074C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:32.943" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 274.948599] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:33 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 40 gitlab # [ 274.949034] gitlab-workhorse[825]: time="2026-11-10T12:01:33Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 274.949462] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:33 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 48 gitlab # [ 274.949983] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f0698f660ab09090b37bde2","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f0698f660ab09090b37bde2: done: 0.018287 sec","job_status":"done","queue_duration_s":1.011305,"scheduling_latency_s":1.011319,"redis_calls":13,"redis_duration_s":0.002584,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000726,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018287,"completed_at":"2026-11-10T12:01:31.787Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.988299] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f35a1460179ae96d7a5c103c","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f35a1460179ae96d7a5c103c: start","job_status":"start","queue_duration_s":1.031658,"scheduling_latency_s":1.031702} gitlab # [ 274.989062] gitaly[919]: time="2026-11-10T12:01:32.950Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NBF7498F8J234B290074C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:32.943" grpc.time_ms=7.456 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 274.989752] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.810Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f35a1460179ae96d7a5c103c","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f35a1460179ae96d7a5c103c: done: 0.021368 sec","job_status":"done","queue_duration_s":1.031658,"scheduling_latency_s":1.031702,"redis_calls":13,"redis_duration_s":0.0024749999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001687,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021368,"completed_at":"2026-11-10T12:01:31.810Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.033257] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86a71ca76b52849d33fd88ca","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86a71ca76b52849d33fd88ca: start","job_status":"start","queue_duration_s":1.054897,"scheduling_latency_s":1.054929} gitlab # [ 275.033696] gitaly[919]: time="2026-11-10T12:01:33.010Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72NBF9AMH297BJJZ4HQ5ZB4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:33.003" grpc.time_ms=7.152 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 275.045521] gitaly[919]: time="2026-11-10T12:01:33.060Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72NBFAMX5RTXJRBZP2BYGF6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:33.054" pid=919 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 275.085351] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86a71ca76b52849d33fd88ca","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86a71ca76b52849d33fd88ca: done: 0.018219 sec","job_status":"done","queue_duration_s":1.054897,"scheduling_latency_s":1.054929,"redis_calls":13,"redis_duration_s":0.002395,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001599,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018219,"completed_at":"2026-11-10T12:01:31.831Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.088696] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d28f840cb1b51e28235336b8","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d28f840cb1b51e28235336b8: start","job_status":"start","queue_duration_s":1.074602,"scheduling_latency_s":1.074629} gitlab # [ 275.099949] gitaly[919]: time="2026-11-10T12:01:33.060Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NBFAMX5RTXJRBZP2BYGF6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:33.054" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 275.139919] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d28f840cb1b51e28235336b8","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d28f840cb1b51e28235336b8: done: 0.011306 sec","job_status":"done","queue_duration_s":1.074602,"scheduling_latency_s":1.074629,"redis_calls":13,"redis_duration_s":0.00163,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011306,"completed_at":"2026-11-10T12:01:31.843Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.146353] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bddeabad6abdb0b1b1072001","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bddeabad6abdb0b1b1072001: start","job_status":"start","queue_duration_s":1.086775,"scheduling_latency_s":1.086789} gitlab # [ 275.185587] gitaly[919]: time="2026-11-10T12:01:33.061Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NBFAMX5RTXJRBZP2BYGF6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T12:01:33.054" grpc.time_ms=6.903 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 275.186420] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bddeabad6abdb0b1b1072001","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bddeabad6abdb0b1b1072001: done: 0.009609 sec","job_status":"done","queue_duration_s":1.086775,"scheduling_latency_s":1.086789,"redis_calls":13,"redis_duration_s":0.001659,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001141,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009609,"completed_at":"2026-11-10T12:01:31.854Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.241228] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"615ea7cd05f02cfb7f9187a4","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-615ea7cd05f02cfb7f9187a4: start","job_status":"start","queue_duration_s":1.098034,"scheduling_latency_s":1.098069} gitlab # [ 275.242172] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"615ea7cd05f02cfb7f9187a4","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-615ea7cd05f02cfb7f9187a4: done: 0.01311 sec","job_status":"done","queue_duration_s":1.098034,"scheduling_latency_s":1.098069,"redis_calls":13,"redis_duration_s":0.0017660000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000625,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001141,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01311,"completed_at":"2026-11-10T12:01:31.868Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.281727] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e6a3ca29359e1b5997aaa38","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e6a3ca29359e1b5997aaa38: start","job_status":"start","queue_duration_s":1.111962,"scheduling_latency_s":1.111975} gitlab # [ 275.282419] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e6a3ca29359e1b5997aaa38","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e6a3ca29359e1b5997aaa38: done: 0.009276 sec","job_status":"done","queue_duration_s":1.111962,"scheduling_latency_s":1.111975,"redis_calls":13,"redis_duration_s":0.001494,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009276,"completed_at":"2026-11-10T12:01:31.879Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.294860] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84f985bc099f930ceb678f65","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84f985bc099f930ceb678f65: start","job_status":"start","queue_duration_s":1.122101,"scheduling_latency_s":1.122115} gitlab # [ 275.306163] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84f985bc099f930ceb678f65","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84f985bc099f930ceb678f65: done: 0.009155 sec","job_status":"done","queue_duration_s":1.122101,"scheduling_latency_s":1.122115,"redis_calls":13,"redis_duration_s":0.001477,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000476,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006029,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009155,"completed_at":"2026-11-10T12:01:31.889Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses gitlab # [ 275.356395] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08dbf6e6b19027d5981a428b","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08dbf6e6b19027d5981a428b: start","job_status":"start","queue_duration_s":1.132122,"scheduling_latency_s":1.132135} gitlab # [ 275.357092] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08dbf6e6b19027d5981a428b","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08dbf6e6b19027d5981a428b: done: 0.009212 sec","job_status":"done","queue_duration_s":1.132122,"scheduling_latency_s":1.132135,"redis_calls":13,"redis_duration_s":0.001514,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00048,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009212,"completed_at":"2026-11-10T12:01:31.899Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.404789] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51068daedb219ef980bb5474","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51068daedb219ef980bb5474: start","job_status":"start","queue_duration_s":1.14218,"scheduling_latency_s":1.142193} gitlab # [ 275.405672] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.909Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51068daedb219ef980bb5474","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51068daedb219ef980bb5474: done: 0.009136 sec","job_status":"done","queue_duration_s":1.14218,"scheduling_latency_s":1.142193,"redis_calls":13,"redis_duration_s":0.001516,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009136,"completed_at":"2026-11-10T12:01:31.909Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.452111] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a7f0e943abe8afb4081cb62","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a7f0e943abe8afb4081cb62: start","job_status":"start","queue_duration_s":1.152171,"scheduling_latency_s":1.152184} gitlab # [ 275.453121] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a7f0e943abe8afb4081cb62","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a7f0e943abe8afb4081cb62: done: 0.009183 sec","job_status":"done","queue_duration_s":1.152171,"scheduling_latency_s":1.152184,"redis_calls":13,"redis_duration_s":0.0015350000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009183,"completed_at":"2026-11-10T12:01:31.919Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.514809] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f849b1118875c23709fc8d1","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f849b1118875c23709fc8d1: start","job_status":"start","queue_duration_s":1.162203,"scheduling_latency_s":1.162217} gitlab # [ 275.515896] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.931Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f849b1118875c23709fc8d1","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f849b1118875c23709fc8d1: done: 0.010845 sec","job_status":"done","queue_duration_s":1.162203,"scheduling_latency_s":1.162217,"redis_calls":13,"redis_duration_s":0.001895,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000692,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010845,"completed_at":"2026-11-10T12:01:31.930Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.555216] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.931Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd2063a2aa1dd711403c90b2","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd2063a2aa1dd711403c90b2: start","job_status":"start","queue_duration_s":1.173917,"scheduling_latency_s":1.173931} gitlab # [ 275.556190] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd2063a2aa1dd711403c90b2","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd2063a2aa1dd711403c90b2: done: 0.009347 sec","job_status":"done","queue_duration_s":1.173917,"scheduling_latency_s":1.173931,"redis_calls":13,"redis_duration_s":0.001611,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009347,"completed_at":"2026-11-10T12:01:31.941Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.597964] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75e2670edf46d87916de80fa","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75e2670edf46d87916de80fa: start","job_status":"start","queue_duration_s":1.1841,"scheduling_latency_s":1.184113} gitlab # [ 275.598716] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75e2670edf46d87916de80fa","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75e2670edf46d87916de80fa: done: 0.009093 sec","job_status":"done","queue_duration_s":1.1841,"scheduling_latency_s":1.184113,"redis_calls":13,"redis_duration_s":0.001479,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00048,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009093,"completed_at":"2026-11-10T12:01:31.951Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.637312] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1478e78ef2a5b6e35ac9e0d5","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1478e78ef2a5b6e35ac9e0d5: start","job_status":"start","queue_duration_s":1.194018,"scheduling_latency_s":1.194031} gitlab # [ 275.638603] gitaly[919]: time="2026-11-10T12:01:33.713Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72NBFVWTAZ0ZYSHXMC5YQWV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T12:02:03.611" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T12:01:33.712" grpc.time_ms=0.642 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 275.639453] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1478e78ef2a5b6e35ac9e0d5","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1478e78ef2a5b6e35ac9e0d5: done: 0.011614 sec","job_status":"done","queue_duration_s":1.194018,"scheduling_latency_s":1.194031,"redis_calls":13,"redis_duration_s":0.0018030000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001141,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011614,"completed_at":"2026-11-10T12:01:31.963Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.677801] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41d84c5e2906f282ca3b1fe0","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41d84c5e2906f282ca3b1fe0: start","job_status":"start","queue_duration_s":1.206716,"scheduling_latency_s":1.206747} gitlab # [ 275.680558] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"41d84c5e2906f282ca3b1fe0","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-41d84c5e2906f282ca3b1fe0: done: 0.009898 sec","job_status":"done","queue_duration_s":1.206716,"scheduling_latency_s":1.206747,"redis_calls":13,"redis_duration_s":0.001526,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009898,"completed_at":"2026-11-10T12:01:31.974Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.734884] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"780e0527621ab6484e4baa54","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-780e0527621ab6484e4baa54: start","job_status":"start","queue_duration_s":1.217461,"scheduling_latency_s":1.217475} gitlab # [ 275.735715] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.984Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"780e0527621ab6484e4baa54","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-780e0527621ab6484e4baa54: done: 0.009191 sec","job_status":"done","queue_duration_s":1.217461,"scheduling_latency_s":1.217475,"redis_calls":13,"redis_duration_s":0.0014919999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009191,"completed_at":"2026-11-10T12:01:31.984Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.774738] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6525c57a3f46f1ed4ae7a22","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6525c57a3f46f1ed4ae7a22: start","job_status":"start","queue_duration_s":1.227512,"scheduling_latency_s":1.227526} gitlab # [ 275.775612] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6525c57a3f46f1ed4ae7a22","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6525c57a3f46f1ed4ae7a22: done: 0.009154 sec","job_status":"done","queue_duration_s":1.227512,"scheduling_latency_s":1.227526,"redis_calls":13,"redis_duration_s":0.001474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000479,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009154,"completed_at":"2026-11-10T12:01:31.994Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.841375] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:31.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57cff780024f9c568164609c","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57cff780024f9c568164609c: start","job_status":"start","queue_duration_s":1.237503,"scheduling_latency_s":1.237518} gitlab # [ 275.842185] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.004Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"57cff780024f9c568164609c","created_at":"2026-11-10T12:01:30.433Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-404746ae97a840dc","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"964801adc352ca284989849247837dc4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.757Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-57cff780024f9c568164609c: done: 0.009125 sec","job_status":"done","queue_duration_s":1.237503,"scheduling_latency_s":1.237518,"redis_calls":13,"redis_duration_s":0.001483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009125,"completed_at":"2026-11-10T12:01:32.004Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.881633] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.005Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c36760b505fc67de8bac8612","created_at":"2026-11-10T12:01:30.751Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ff663d6a6fae3506d1837f8e6a0ab57","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.770Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c36760b505fc67de8bac8612: start","job_status":"start","queue_duration_s":1.235007,"scheduling_latency_s":1.235035} gitlab # [ 275.882199] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.015Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 275.882817] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.020Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c36760b505fc67de8bac8612","created_at":"2026-11-10T12:01:30.751Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ff663d6a6fae3506d1837f8e6a0ab57","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.770Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c36760b505fc67de8bac8612: done: 0.014326 sec","job_status":"done","queue_duration_s":1.235007,"scheduling_latency_s":1.235035,"redis_calls":9,"redis_duration_s":0.001215,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000286,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000667,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000262,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014326,"completed_at":"2026-11-10T12:01:32.020Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000531,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.932301] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.021Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"18084361b9cfc4f8b6d16fc8","created_at":"2026-11-10T12:01:30.792Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f015b4231db5673ee04e71626e1059d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.801Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-18084361b9cfc4f8b6d16fc8: start","job_status":"start","queue_duration_s":1.219325,"scheduling_latency_s":1.219339} gitlab # [ 275.932929] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.027Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 275.933597] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.031Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"18084361b9cfc4f8b6d16fc8","created_at":"2026-11-10T12:01:30.792Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f015b4231db5673ee04e71626e1059d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.801Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-18084361b9cfc4f8b6d16fc8: done: 0.010959 sec","job_status":"done","queue_duration_s":1.219325,"scheduling_latency_s":1.219339,"redis_calls":9,"redis_duration_s":0.00115,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000282,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000239,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010959,"completed_at":"2026-11-10T12:01:32.031Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000269,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 275.958787] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"99f647922589f7e4febf230e","created_at":"2026-11-10T12:01:30.806Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1ee36850181e5902a2d16da0ae79d7e2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T12:01:30.822Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-99f647922589f7e4febf230e: start","job_status":"start","queue_duration_s":1.21021,"scheduling_latency_s":1.210224} gitlab # [ 276.003271] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.039Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 276.004076] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.043Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"99f647922589f7e4febf230e","created_at":"2026-11-10T12:01:30.806Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1ee36850181e5902a2d16da0ae79d7e2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T12:01:30.822Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-99f647922589f7e4febf230e: done: 0.010975 sec","job_status":"done","queue_duration_s":1.21021,"scheduling_latency_s":1.210224,"redis_calls":9,"redis_duration_s":0.001307,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000254,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00081,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000243,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010975,"completed_at":"2026-11-10T12:01:32.043Z","load_balancing_strategy":"primary","db_duration_s":0.000273,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.042750] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.044Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0610196487681e31b05cd1e2","created_at":"2026-11-10T12:01:30.829Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf3145b448faf517b7bac9bc79c56dbe","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.837Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0610196487681e31b05cd1e2: start","job_status":"start","queue_duration_s":1.206812,"scheduling_latency_s":1.206826} gitlab # [ 276.043704] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.063Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0610196487681e31b05cd1e2","created_at":"2026-11-10T12:01:30.829Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cf3145b448faf517b7bac9bc79c56dbe","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.837Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0610196487681e31b05cd1e2: done: 0.018308 sec","job_status":"done","queue_duration_s":1.206812,"scheduling_latency_s":1.206826,"redis_calls":12,"redis_duration_s":0.001631,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000782,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000589,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.005762185,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.018308,"completed_at":"2026-11-10T12:01:32.063Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001803,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 276.087689] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.063Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"bb598b3dfbc91b8bc7b50b05","created_at":"2026-11-10T12:01:30.845Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"179c1894ee58c35398627ab4957d76cb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.850Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-bb598b3dfbc91b8bc7b50b05: start","job_status":"start","queue_duration_s":1.213503,"scheduling_latency_s":1.213517} gitlab # [ 276.088470] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.071Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 276.089036] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.075Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"bb598b3dfbc91b8bc7b50b05","created_at":"2026-11-10T12:01:30.845Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"179c1894ee58c35398627ab4957d76cb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.850Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-bb598b3dfbc91b8bc7b50b05: done: 0.011407 sec","job_status":"done","queue_duration_s":1.213503,"scheduling_latency_s":1.213517,"redis_calls":9,"redis_duration_s":0.00124,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000714,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011407,"completed_at":"2026-11-10T12:01:32.075Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000278,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.114913] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.076Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"0bbd108c11402a1e480be0a3","created_at":"2026-11-10T12:01:30.852Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5759153fda061f4e58ac40c9afa216b0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.855Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-0bbd108c11402a1e480be0a3: start","job_status":"start","queue_duration_s":1.220702,"scheduling_latency_s":1.220716} gitlab # [ 276.162347] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.099Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"0bbd108c11402a1e480be0a3","created_at":"2026-11-10T12:01:30.852Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5759153fda061f4e58ac40c9afa216b0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.855Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-0bbd108c11402a1e480be0a3: done: 0.022997 sec","job_status":"done","queue_duration_s":1.220702,"scheduling_latency_s":1.220716,"redis_calls":11,"redis_duration_s":0.001528,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000643,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000643,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000382731,"exclusive_lock_hold_duration_s":0.007913576,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.022997,"completed_at":"2026-11-10T12:01:32.099Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006933,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.167672] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.100Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"25a72ec040a487256e4ca77c","created_at":"2026-11-10T12:01:30.864Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b54bf8eb2575111837d5ca62e0e5e8f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.870Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-25a72ec040a487256e4ca77c: start","job_status":"start","queue_duration_s":1.229346,"scheduling_latency_s":1.229361} gitlab # [ 276.208676] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.219Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"25a72ec040a487256e4ca77c","created_at":"2026-11-10T12:01:30.864Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b54bf8eb2575111837d5ca62e0e5e8f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.870Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-25a72ec040a487256e4ca77c: done: 0.118891 sec","job_status":"done","queue_duration_s":1.229346,"scheduling_latency_s":1.229361,"redis_calls":11,"redis_duration_s":0.001675,"redis_read_bytes":9,"redis_write_bytes":1292,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000689,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000733,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.094914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00031261,"exclusive_lock_hold_duration_s":0.105196889,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.118891,"completed_at":"2026-11-10T12:01:32.219Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027215,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.219038] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.220Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d429ac65b8da869364010291","created_at":"2026-11-10T12:01:30.877Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6eb857806bea1e5bd10155d7aac92192","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T12:01:30.880Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d429ac65b8da869364010291: start","job_status":"start","queue_duration_s":1.340117,"scheduling_latency_s":1.340133} gitlab # [ 276.258369] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.251Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 276.259063] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.256Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d429ac65b8da869364010291","created_at":"2026-11-10T12:01:30.877Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-986c621a37ad475f","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6eb857806bea1e5bd10155d7aac92192","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T12:01:30.880Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d429ac65b8da869364010291: done: 0.036005 sec","job_status":"done","queue_duration_s":1.340117,"scheduling_latency_s":1.340133,"redis_calls":9,"redis_duration_s":0.004265,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00296,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001041,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000264,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036005,"completed_at":"2026-11-10T12:01:32.256Z","load_balancing_strategy":"primary","db_duration_s":0.020164,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.270267] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.257Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8301f59ea6aaa01d61950318","created_at":"2026-11-10T12:01:30.905Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-c244593e6a9042de","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0710c9712329a3cbed6f05749ceb2671","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.912Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8301f59ea6aaa01d61950318: start","job_status":"start","queue_duration_s":1.344674,"scheduling_latency_s":1.344691} gitlab # [ 276.281539] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.267Z","message":"No existing merge request to be cleaned up."} gitlab # [ 276.282980] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.276Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8301f59ea6aaa01d61950318","created_at":"2026-11-10T12:01:30.905Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-c244593e6a9042de","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0710c9712329a3cbed6f05749ceb2671","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.912Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8301f59ea6aaa01d61950318: done: 0.018778 sec","job_status":"done","queue_duration_s":1.344674,"scheduling_latency_s":1.344691,"redis_calls":12,"redis_duration_s":0.001839,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018778,"completed_at":"2026-11-10T12:01:32.276Z","load_balancing_strategy":"primary","db_duration_s":0.002782,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.325082] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.277Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d377bd5ea0d1a45374f3d1d3","created_at":"2026-11-10T12:01:30.905Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-c244593e6a9042de","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0710c9712329a3cbed6f05749ceb2671","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.912Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d377bd5ea0d1a45374f3d1d3: start","job_status":"start","queue_duration_s":1.364398,"scheduling_latency_s":1.364413} gitlab # [ 276.329452] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.284Z","message":"No existing merge request to be cleaned up."} gitlab # [ 276.330889] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.289Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d377bd5ea0d1a45374f3d1d3","created_at":"2026-11-10T12:01:30.905Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-c244593e6a9042de","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0710c9712329a3cbed6f05749ceb2671","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.912Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d377bd5ea0d1a45374f3d1d3: done: 0.012512 sec","job_status":"done","queue_duration_s":1.364398,"scheduling_latency_s":1.364413,"redis_calls":10,"redis_duration_s":0.001327,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000791,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012512,"completed_at":"2026-11-10T12:01:32.289Z","load_balancing_strategy":"primary","db_duration_s":0.002475,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.376903] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.290Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e35bea1bf7652c101eb01676","created_at":"2026-11-10T12:01:30.905Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-c244593e6a9042de","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0710c9712329a3cbed6f05749ceb2671","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.912Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e35bea1bf7652c101eb01676: start","job_status":"start","queue_duration_s":1.377789,"scheduling_latency_s":1.377804} gitlab # [ 276.377684] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.296Z","message":"No existing merge request to be cleaned up."} gitlab # [ 276.378195] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.301Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e35bea1bf7652c101eb01676","created_at":"2026-11-10T12:01:30.905Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-c244593e6a9042de","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0710c9712329a3cbed6f05749ceb2671","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.912Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e35bea1bf7652c101eb01676: done: 0.011222 sec","job_status":"done","queue_duration_s":1.377789,"scheduling_latency_s":1.377804,"redis_calls":10,"redis_duration_s":0.0012699999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000762,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011222,"completed_at":"2026-11-10T12:01:32.301Z","load_balancing_strategy":"primary","db_duration_s":0.000995,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.419382] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.303Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0b955bf8e874d104865bd4d4","created_at":"2026-11-10T12:01:30.905Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-c244593e6a9042de","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0710c9712329a3cbed6f05749ceb2671","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.912Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0b955bf8e874d104865bd4d4: start","job_status":"start","queue_duration_s":1.390802,"scheduling_latency_s":1.390848} gitlab # [ 276.421843] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.314Z","message":"No existing merge request to be cleaned up."} gitlab # [ 276.423774] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.321Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0b955bf8e874d104865bd4d4","created_at":"2026-11-10T12:01:30.905Z","trace_propagation_headers":{"sentry-trace":"11d39367621642fa9f2399c4fef2d97e-c244593e6a9042de","baggage":"sentry-trace_id=11d39367621642fa9f2399c4fef2d97e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"0710c9712329a3cbed6f05749ceb2671","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T12:01:30.912Z","job_size_bytes":2,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0b955bf8e874d104865bd4d4: done: 0.017587 sec","job_status":"done","queue_duration_s":1.390802,"scheduling_latency_s":1.390848,"redis_calls":10,"redis_duration_s":0.001474,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000616,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000858,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017587,"completed_at":"2026-11-10T12:01:32.320Z","load_balancing_strategy":"primary","db_duration_s":0.001488,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.475122] sidekiq-cluster[2616]: {"severity":"INFO","time":"2026-11-10T12:01:32.322Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"2768b544cbf6656ef0aea9ac","created_at":"2026-11-10T12:01:31.155Z","trace_propagation_headers":{"sentry-trace":"63cdc3dba052401ca4c87b3d29ed4b77-304ba8cd6db44e5f","baggage":"sentry-trace_id=63cdc3dba052401ca4c87b3d29ed4b77,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72NBD2Y5VG3MVZJSDBY97BW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-10T12:01:31.158Z","job_size_bytes":3,"pid":2616,"sidekiq_tid":"m830","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2768b544cbf6656ef0aea9ac: start","job_status":"start","queue_duration_s":1.16335,"scheduling_latency_s":1.163365} gitlab # [ 276.537105] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/10:12:01:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 1103 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/6279a8528aa5cc381d8ddfcd046a8d644342b5aa/statuses, in 1.18 seconds) (finished: subtest: eval and build post commit statuses, in 8.20 seconds) (finished: run the VM test script, in 333.45 seconds) test script finished in 333.49s cleanup kill QemuMachine (pid 46) gitlab # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14) (finished: cleanup, in 1.46 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-11-10T12:00:30.719Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '6279a8528aa5cc381d8ddfcd046a8d644342b5aa', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-11-10T12:01:25.557551Z', 'started_at': '2026-11-10T12:01:26.060165Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}] [{'id': 5, 'sha': '6279a8528aa5cc381d8ddfcd046a8d644342b5aa', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T12:01:25.961Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '6279a8528aa5cc381d8ddfcd046a8d644342b5aa', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T12:01:25.961Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '6279a8528aa5cc381d8ddfcd046a8d644342b5aa', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T12:01:25.961Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '6279a8528aa5cc381d8ddfcd046a8d644342b5aa', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T12:01:25.961Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '6279a8528aa5cc381d8ddfcd046a8d644342b5aa', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 4s (1 warning)', 'created_at': '2026-11-10T12:01:25.961Z', 'started_at': None, 'finished_at': '2026-11-10T12:01:31.147Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '6279a8528aa5cc381d8ddfcd046a8d644342b5aa', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 4s (1 warning)', 'created_at': '2026-11-10T12:01:25.961Z', 'started_at': None, 'finished_at': '2026-11-10T12:01:31.147Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '6279a8528aa5cc381d8ddfcd046a8d644342b5aa', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 4s (1 warning)', 'created_at': '2026-11-10T12:01:25.961Z', 'started_at': None, 'finished_at': '2026-11-10T12:01:31.147Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '6279a8528aa5cc381d8ddfcd046a8d644342b5aa', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-11-10T12:01:32.452Z', 'started_at': None, 'finished_at': '2026-11-10T12:01:32.607Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]