tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab: QEMU running (pid 46) gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab # Formatting '/build/vm-state-gitlab/tmp.XzVi12WU3t', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) (finished: start all VMs, in 53.17 seconds) gitlab # Discarding device blocks: 0/262144 done gitlab: waiting for unit setup-gitlab.service gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab: waiting for the VM to finish booting gitlab # Filesystem UUID: ea38587d-7a99-4321-ac57-295a3bb58a19 gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/idc42pxlv8gqw6zca87f180b1djz2nf1-nixos-system-gitlab-test/init regInfo=/nix/store/4kdmj68pgnpli788995j1yrpddzmsyy2-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000000] kvm-clock: using sched offset of 517523395 cycles gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000005] tsc: Detected 2400.012 MHz processor gitlab # [ 0.000811] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000879] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002732] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002744] Using GB pages for direct mapping gitlab # [ 0.002831] RAMDISK: [mem 0xbe31c000-0xbffcffff] gitlab # [ 0.002839] ACPI: Early table checksum verification disabled gitlab # [ 0.002842] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002845] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002850] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002857] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002859] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.002860] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002862] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002863] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002864] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.002865] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.002866] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.002866] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.002867] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.002867] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003093] No NUMA configuration found gitlab # [ 0.003094] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003096] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff] gitlab # [ 0.005343] Zone ranges: gitlab # [ 0.005343] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.005345] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.005346] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005347] Device empty gitlab # [ 0.005348] Movable zone start for each node gitlab # [ 0.005348] Early memory node ranges gitlab # [ 0.005349] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.005350] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.005351] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005352] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.005371] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.005632] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.112583] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.113000] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.113015] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.113049] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.113053] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.113055] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.113056] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.113057] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.113058] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.113060] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.113061] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.113066] TSC deadline timer available gitlab # [ 0.113071] CPU topo: Max. logical packages: 1 gitlab # [ 0.113072] CPU topo: Max. logical dies: 1 gitlab # [ 0.113072] CPU topo: Max. dies per package: 1 gitlab # [ 0.113076] CPU topo: Max. threads per core: 1 gitlab # [ 0.113076] CPU topo: Num. cores per package: 4 gitlab # [ 0.113077] CPU topo: Num. threads per package: 4 gitlab # [ 0.113077] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.113101] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.113117] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.113120] kvm-guest: setup PV sched yield gitlab # [ 0.113135] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.113137] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.113138] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.113139] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.113141] Booting paravirtualized kernel on KVM gitlab # [ 0.113145] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.117606] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.120058] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.120111] kvm-guest: PV spinlocks enabled gitlab # [ 0.120113] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.120116] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/idc42pxlv8gqw6zca87f180b1djz2nf1-nixos-system-gitlab-test/init regInfo=/nix/store/4kdmj68pgnpli788995j1yrpddzmsyy2-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.120216] Unknown kernel command line parameters "regInfo=/nix/store/4kdmj68pgnpli788995j1yrpddzmsyy2-closure-info/registration", will be passed to user space. gitlab # [ 0.120229] random: crng init done gitlab # [ 0.120230] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.129003] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.133273] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.133347] software IO TLB: area num 4. gitlab # [ 0.206545] Fallback order for Node 0: 0 gitlab # [ 0.206553] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.206554] Policy zone: Normal gitlab # [ 0.209281] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.218715] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.232042] allocated 12582912 bytes of page_ext gitlab # [ 0.241866] ftrace: allocating 48717 entries in 192 pages gitlab # [ 0.241868] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.242714] Dynamic Preempt: lazy gitlab # [ 0.242888] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.242889] rcu: RCU event tracing is enabled. gitlab # [ 0.242890] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.242891] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.242892] Rude variant of Tasks RCU enabled. gitlab # [ 0.242892] Tracing variant of Tasks RCU enabled. gitlab # [ 0.242892] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.242893] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.242908] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.242910] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.242911] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.247607] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.247933] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.247942] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.248054] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.251655] Console: colour VGA+ 80x25 gitlab # [ 0.251658] printk: legacy console [tty0] enabled gitlab # [ 0.282550] printk: legacy console [ttyS0] enabled gitlab # [ 0.391821] ACPI: Core revision 20250807 gitlab # [ 0.392788] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.394389] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.395458] x2apic enabled gitlab # [ 0.396220] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.397187] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.398464] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.399960] kvm-guest: setup PV IPIs gitlab # [ 0.401571] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.402630] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 0.404442] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) gitlab # [ 0.405527] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.406576] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.408439] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.409443] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.410439] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.411440] Transient Scheduler Attacks: Vulnerable: No microcode gitlab # [ 0.413236] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.414204] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.415259] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.416447] Spectre V2 : Enabling IBPB for BPF gitlab # [ 0.417237] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.418440] active return thunk: srso_alias_return_thunk gitlab # [ 0.420156] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.421438] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.422285] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.423439] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.424439] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.425439] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.426439] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.427440] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.429440] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.430439] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.431438] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.432439] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.433439] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. gitlab # [ 0.462177] Freeing SMP alternatives memory: 44K gitlab # [ 0.462438] pid_max: default: 32768 minimum: 301 gitlab # [ 0.463536] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.464558] landlock: Up and running. gitlab # [ 0.465242] Yama: becoming mindful. gitlab # [ 0.466652] LSM support for eBPF active gitlab # [ 0.467666] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.468591] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.471732] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.472996] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.473449] ... version: 2 gitlab # [ 0.474228] ... bit width: 48 gitlab # [ 0.474442] ... generic counters: 6 gitlab # [ 0.475191] ... generic bitmap: 000000000000003f gitlab # [ 0.475442] ... fixed-purpose counters: 0 gitlab # [ 0.476209] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.476441] ... value mask: 0000ffffffffffff gitlab # [ 0.477414] ... max period: 00007fffffffffff gitlab # [ 0.478193] ... global_ctrl mask: 000000000000003f gitlab # [ 0.478544] signal: max sigframe size: 3376 gitlab # [ 0.479380] rcu: Hierarchical SRCU implementation. gitlab # [ 0.480100] rcu: Max phase no-delay instances is 400. gitlab # [ 0.480629] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.485910] smp: Bringing up secondary CPUs ... gitlab # [ 0.486834] smpboot: x86: Booting SMP configuration: gitlab # [ 0.487445] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.490579] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.492162] smpboot: Total of 4 processors activated (19200.09 BogoMIPS) gitlab # [ 0.492782] Memory: 6011320K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260100K reserved, 0K cma-reserved) gitlab # [ 0.495442] devtmpfs: initialized gitlab # [ 0.496253] x86/mm: Memory block size: 128MB gitlab # [ 0.499507] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.500519] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.501542] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.502735] PM: RTC time: 10:33:44, date: 2026-11-13 gitlab # [ 0.506301] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.508586] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.509537] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.511538] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.512453] audit: initializing netlink subsys (disabled) gitlab # [ 0.513498] audit: type=2000 audit(1794566024.522:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.513705] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.515442] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.516441] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.517443] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.518441] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.519478] cpuidle: using governor menu gitlab # [ 0.521885] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.522685] PCI: Using configuration type 1 for base access gitlab # [ 0.523443] PCI: Using configuration type 1 for extended access gitlab # [ 0.524636] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.527562] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.530452] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.532448] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.535447] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.542093] ACPI: Added _OSI(Module Device) gitlab # [ 0.542444] ACPI: Added _OSI(Processor Device) gitlab # [ 0.543188] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.545335] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.549348] ACPI: Interpreter enabled gitlab # [ 0.551524] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.553450] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.555545] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.559451] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.561921] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.574653] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.577458] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.582422] acpiphp: Slot [3] registered gitlab # [ 0.583515] acpiphp: Slot [4] registered gitlab # [ 0.585531] acpiphp: Slot [5] registered gitlab # [ 0.587509] acpiphp: Slot [6] registered gitlab # [ 0.589494] acpiphp: Slot [7] registered gitlab # [ 0.590507] acpiphp: Slot [8] registered gitlab # [ 0.591493] acpiphp: Slot [9] registered gitlab # [ 0.593491] acpiphp: Slot [10] registered gitlab # [ 0.594493] acpiphp: Slot [11] registered gitlab # [ 0.596513] acpiphp: Slot [12] registered gitlab # [ 0.597502] acpiphp: Slot [13] registered gitlab # [ 0.598483] acpiphp: Slot [14] registered gitlab # [ 0.600484] acpiphp: Slot [15] registered gitlab # [ 0.601500] acpiphp: Slot [16] registered gitlab # [ 0.603487] acpiphp: Slot [17] registered gitlab # [ 0.604483] acpiphp: Slot [18] registered gitlab # [ 0.606483] acpiphp: Slot [19] registered gitlab # [ 0.607479] acpiphp: Slot [20] registered gitlab # [ 0.608256] acpiphp: Slot [21] registered gitlab # [ 0.609021] acpiphp: Slot [22] registered gitlab # [ 0.609470] acpiphp: Slot [23] registered gitlab # [ 0.610197] acpiphp: Slot [24] registered gitlab # [ 0.611462] acpiphp: Slot [25] registered gitlab # [ 0.612213] acpiphp: Slot [26] registered gitlab # [ 0.612470] acpiphp: Slot [27] registered gitlab # [ 0.613923] acpiphp: Slot [28] registered gitlab # [ 0.615505] acpiphp: Slot [29] registered gitlab # [ 0.617503] acpiphp: Slot [30] registered gitlab # [ 0.619525] acpiphp: Slot [31] registered gitlab # [ 0.620639] PCI host bridge to bus 0000:00 gitlab # [ 0.622478] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.625449] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.628449] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.631447] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.634448] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.637474] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.640083] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.644890] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.649503] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.656533] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.658514] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.661446] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.663445] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.665446] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.667849] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.672517] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.676563] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.678890] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.680453] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.683594] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.708590] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.711481] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.714501] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.718207] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.723246] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.732476] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 0.734458] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 0.736488] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 0.756451] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 0.759032] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.763479] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 0.764448] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 0.765463] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 0.767971] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.772450] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 0.773387] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 0.775235] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 0.777646] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.781471] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 0.783449] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 0.784464] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 0.786662] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.790471] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 0.791396] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 0.792269] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 0.795754] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.798362] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 0.800159] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 0.800464] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 0.804251] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.806469] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 0.807388] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 0.809254] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 0.810441] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 0.812648] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.814933] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 0.816464] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 0.818686] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.822481] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 0.823409] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 0.824258] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 0.827965] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.832480] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 0.833421] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 0.834301] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 0.840267] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 0.841402] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 0.842395] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 0.844386] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 0.845333] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 0.848455] iommu: Default domain type: Translated gitlab # [ 0.849334] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 0.850732] ACPI: bus type USB registered gitlab # [ 0.851506] usbcore: registered new interface driver usbfs gitlab # [ 0.852477] usbcore: registered new interface driver hub gitlab # [ 0.853384] usbcore: registered new device driver usb gitlab # [ 0.855276] NetLabel: Initializing gitlab # [ 0.855891] NetLabel: domain hash size = 128 gitlab # [ 0.856442] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 0.857457] NetLabel: unlabeled traffic allowed by default gitlab # [ 0.858419] PCI: Using ACPI for IRQ routing gitlab # [ 0.859644] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 0.860438] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 0.860438] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 0.863445] vgaarb: loaded gitlab # [ 0.864179] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 0.864441] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 0.869524] clocksource: Switched to clocksource kvm-clock gitlab # [ 0.871066] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.871829] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.873364] pnp: PnP ACPI init gitlab # [ 0.874527] pnp: PnP ACPI: found 6 devices gitlab # [ 0.884886] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 0.888638] clocksource: Switched to clocksource acpi_pm gitlab # [ 0.891275] NET: Registered PF_INET protocol family gitlab # [ 0.896347] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.931303] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.932887] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.934317] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.937991] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.939372] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.940880] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.942540] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.943989] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.945386] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.946405] NET: Registered PF_XDP protocol family gitlab # [ 0.947280] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 0.948342] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 0.949402] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 0.950584] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.951753] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.953132] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 0.954161] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 0.956708] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 0.959248] PCI: CLS 0 bytes, default 64 gitlab # [ 0.960087] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 0.960768] Trying to unpack rootfs image as initramfs... gitlab # [ 0.961198] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB) gitlab # [ 0.964040] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 0.984854] Initialise system trusted keyrings gitlab # [ 0.986095] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 0.997733] Key type asymmetric registered gitlab # [ 0.998454] Asymmetric key parser 'x509' registered gitlab # [ 0.999387] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 1.001177] io scheduler mq-deadline registered gitlab # [ 1.002019] io scheduler kyber registered gitlab # [ 1.007080] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 1.008430] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 1.010950] Linux agpgart interface v0.103 gitlab # [ 1.011757] ACPI: bus type drm_connector registered gitlab # [ 1.015170] usbcore: registered new interface driver usbserial_generic gitlab # [ 1.016347] usbserial: USB Serial support registered for generic gitlab # [ 1.017382] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 1.017382] Please enable it if your BIOS has the CPPC option. gitlab # [ 1.019747] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 1.021208] drop_monitor: Initializing network drop monitor service gitlab # [ 1.022422] NET: Registered PF_INET6 protocol family gitlab # [ 1.024183] Segment Routing with IPv6 gitlab # [ 1.024916] In-situ OAM (IOAM) with IPv6 gitlab # [ 1.027049] IPI shorthand broadcast: enabled gitlab # [ 1.031452] sched_clock: Marking stable (882010320, 148496618)->(1054363508, -23856570) gitlab # [ 1.033315] registered taskstats version 1 gitlab # [ 1.034371] Loading compiled-in X.509 certificates gitlab # [ 1.043901] Demotion targets for Node 0: null gitlab # [ 1.045090] Key type .fscrypt registered gitlab # [ 1.045831] Key type fscrypt-provisioning registered gitlab # [ 1.046824] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 1.047793] ima: Allocated hash algorithm: sha1 gitlab # [ 1.048646] ima: No architecture policies found gitlab # [ 1.049744] PM: Magic number: 2:98:578 gitlab # [ 1.050458] platform pcspkr: hash matches gitlab # [ 1.052093] RAS: Correctable Errors collector initialized. gitlab # [ 1.056608] clk: Disabling unused clocks gitlab # [ 1.057309] PM: genpd: Disabling unused power domains gitlab # [ 1.241656] Freeing initrd memory: 29392K gitlab # [ 1.248374] Freeing unused decrypted memory: 2028K gitlab # [ 1.254671] Freeing unused kernel image (initmem) memory: 3640K gitlab # [ 1.257335] Write protecting the kernel read-only data: 32768k gitlab # [ 1.262138] Freeing unused kernel image (text/rodata gap) memory: 1220K gitlab # [ 1.266063] Freeing unused kernel image (rodata/data gap) memory: 760K gitlab # [ 1.335233] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.336396] Run /init as init process gitlab # [ 1.347607] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.368751] fuse: init (API version 7.45) gitlab # [ 1.377135] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 1.383914] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 1.387111] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 1.425242] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 1.761360] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.802024] systemd[1]: Detected virtualization kvm. gitlab # [ 1.804250] systemd[1]: Detected architecture x86-64. gitlab # [ 1.806458] systemd[1]: Running in initrd. gitlab # [ 1.808996] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.811925] systemd[1]: Hostname set to . gitlab # [ 1.930048] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 1.974070] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 1.985116] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 1.986334] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 1.987809] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 1.989033] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 1.990193] systemd[1]: Reached target Path Units. gitlab # [ 1.991106] systemd[1]: Reached target Slice Units. gitlab # [ 1.992049] systemd[1]: Reached target Swaps. gitlab # [ 1.992904] systemd[1]: Reached target Timer Units. gitlab # [ 1.993922] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 1.995191] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 1.996396] systemd[1]: Listening on Journal Sockets. gitlab # [ 1.997422] systemd[1]: Listening on udev Control Socket. gitlab # [ 1.998434] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 1.999396] systemd[1]: Reached target Socket Units. gitlab # [ 2.001379] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 2.003638] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 2.005731] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 2.013888] systemd[1]: Starting Journal Service... gitlab # [ 2.018033] systemd[1]: Starting Load Kernel Modules... gitlab # [ 2.020589] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 2.024818] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 2.034700] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 2.043065] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.048148] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.052908] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.054676] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.056053] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.060065] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 2.064612] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.070957] systemd-journald[91]: Collecting audit messages is disabled. gitlab # [ 2.075676] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.078716] netfs: FS-Cache loaded gitlab # [ 2.090748] 9pnet: Installing 9P2000 support gitlab # [ 2.095698] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.097668] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.099397] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.101675] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.103469] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 2.113668] systemd[1]: Started Journal Service. gitlab # [ 1.965285] systemd-modules-load[92]: Using 4 probe threads gitlab # [ 1.967369] systemd-modules-load[92]: Inserted module 'virtio_balloon' gitlab # [ 1.968495] systemd-modules-load[92]: Inserted module 'virtio_gpu' gitlab # [ 1.969523] systemd-modules-load[92]: Inserted module 'dm_mod' gitlab # [ 1.970489] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 1.971579] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 1.972679] systemd[1]: Reached target Local File Systems. gitlab # [ 1.973761] systemd[1]: Starting Create System Files and Directories... gitlab # [ 1.974890] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 1.987441] systemd[1]: Finished Create System Files and Directories. gitlab # [ 1.998168] systemd-udevd[104]: Using default interface naming scheme 'v261'. gitlab # [ 2.008423] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 2.018155] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.018956] systemd[1]: Reached target System Initialization. gitlab # [ 2.019748] systemd[1]: Reached target Basic System. gitlab # [ 2.265571] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 2.271947] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 2.274351] SCSI subsystem initialized gitlab # [ 2.275600] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 2.299029] virtio_blk virtio5: [vdb] 11953512 512-byte logical blocks (6.12 GB/5.70 GiB) gitlab # [ 2.306119] scsi host0: ata_piix gitlab # [ 2.312183] scsi host1: ata_piix gitlab # [ 2.312913] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 2.314016] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.321409] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 2.323231] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 2.324248] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 2.324479] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 2.324642] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 2.324830] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 2.325130] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 2.328550] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 2.329740] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 2.330776] usb usb1: Product: UHCI Host Controller gitlab # [ 2.331451] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd gitlab # [ 2.332195] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 2.197227] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.357425] hub 1-0:1.0: USB hub found gitlab # [ 2.209142] (udev-worker)[135]: Network inter[ 2.358080] hub 1-0:1.0: 2 ports detected gitlab # face NamePolicy= disabled on kernel command line. gitlab # [ 2.212354] (udev-worker)[119]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.215250] (udev-worker)[119]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.220871] systemd-vconsole-setup[149]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.223915] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.225170] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.228414] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.229334] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.230136] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.253620] systemd-fsck[156]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.257165] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.476227] ata2: found unknown device (class 0) gitlab # [ 2.479254] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 2.483267] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 2.520725] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # [ 2.550838] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 2.578468] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 2.580600] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 2.749982] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 2.753057] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 2.755767] usb 1-1: Product: QEMU USB Tablet gitlab # [ 2.757422] usb 1-1: Manufacturer: QEMU gitlab # [ 2.758950] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 2.804756] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 2.817825] usbcore: registered new interface driver usbhid gitlab # [ 2.819876] usbhid: USB HID core driver gitlab # [ 2.829136] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 2.832381] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 2.886459] systemd[1]: Mounting /sysroot... gitlab # [ 3.189667] EXT4-fs (vda): mounted filesystem ea38587d-7a99-4321-ac57-295a3bb58a19 r/w with ordered data mode. Quota mode: none. gitlab # [ 3.048444] systemd[1]: Mounted /sysroot. gitlab # [ 3.050450] systemd[1]: Reached target Initrd Root File System. gitlab # [ 3.053712] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 3.055735] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 3.064791] systemd[1]: Mounting /sysroot/run... gitlab # [ 3.066523] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.073333] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.075451] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.230015] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.083277] systemd-sysroot-fstab-check[177]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.087144] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.091931] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.093539] systemd[1]: Reload requested from client PID 177 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.096686] systemd[1]: Reloading... gitlab # [ 3.246973] 9p: Installing v9fs 9p2000 file system support gitlab # [ 3.153538] systemd[1]: Reloading finished in 59 ms. gitlab # [ 3.176983] systemd-sysroot-fstab-check[177]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.179975] systemd[1]: Mounted /sysroot/run. gitlab # [ 3.181762] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 3.183821] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.185707] systemd-sysroot-fstab-check[177]: Requesting swap.target/start/replace... gitlab # [ 3.188420] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.190624] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.193106] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.195451] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.203484] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.206234] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.887213] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 3.935060] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 3.936875] systemd[1]: Reached target Initrd File Systems. gitlab # [ 3.939136] systemd[1]: Starting Find NixOS closure... gitlab # [ 3.940875] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 3.971653] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 3.974633] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 3.988714] systemd[1]: Finished Find NixOS closure. gitlab # [ 3.990535] systemd[1]: Reached target Initrd Default Target. gitlab # [ 3.992471] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 4.022577] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 4.024606] systemd[1]: Stopped target Basic System. gitlab # [ 4.026374] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 4.028255] systemd[1]: Stopped target Path Units. gitlab # [ 4.029911] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.032368] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.035436] systemd[1]: Stopped target Slice Units. gitlab # [ 4.039253] systemd[1]: Stopped target Socket Units. gitlab # [ 4.042377] systemd[1]: Stopped target System Initialization. gitlab # [ 4.044820] systemd[1]: Stopped target Swaps. gitlab # [ 4.046388] systemd[1]: Stopped target Timer Units. gitlab # [ 4.048076] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.050463] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.052471] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.054806] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.056305] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 4.059615] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.061464] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.063326] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.064611] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.066400] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.067610] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.069490] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.071348] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.073163] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.074654] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.075966] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.077564] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.079270] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.080118] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.081081] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.082039] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.082739] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.083707] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.084796] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.085773] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.086537] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.087416] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.088437] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.089456] systemd[1]: Closed udev Control Socket. gitlab # [ 4.090206] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.090951] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.091953] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.092779] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.093855] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.094832] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.095772] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.100880] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.101935] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.102678] systemd[1]: Reached target Switch Root. gitlab # [ 4.104306] systemd[1]: Starting NixOS Activation... gitlab # [ 4.197139] initrd-nixos-activation-start[225]: booting system configuration /nix/store/idc42pxlv8gqw6zca87f180b1djz2nf1-nixos-system-gitlab-test gitlab # [ 4.233311] initrd-nixos-activation-start[225]: running activation script... gitlab # [ 4.465236] initrd-nixos-activation-start[249]: setting up /etc... gitlab # [ 4.547774] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.550442] systemd[1]: Finished NixOS Activation. gitlab # [ 4.552589] systemd[1]: Starting Switch Root... gitlab # [ 4.583070] systemd[1]: Switching root. gitlab # [ 4.796213] systemd-journald[91]: Received SIGTERM from PID 1 (systemd). gitlab # [ 4.898917] NET: Registered PF_VSOCK protocol family gitlab # [ 5.262225] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.272417] systemd[1]: Detected virtualization kvm. gitlab # [ 5.274282] systemd[1]: Detected architecture x86-64. gitlab # [ 5.276283] systemd[1]: Detected first boot. gitlab # [ 5.278874] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.436109] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.534167] systemd[1]: Applying preset policy. gitlab # [ 5.711823] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 5.830641] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 5.831989] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 5.834164] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 5.836300] systemd[1]: Created slice Slice /system/getty. gitlab # [ 5.837684] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 5.839100] systemd[1]: Created slice User and Session Slice. gitlab # [ 5.840279] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 5.841538] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 5.842675] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 5.843365] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 5.844203] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 5.845075] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 5.845907] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 5.846753] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 5.847635] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 5.848553] systemd[1]: Reached target Path Units. gitlab # [ 5.849238] systemd[1]: Reached target Remote File Systems. gitlab # [ 5.850068] systemd[1]: Reached target Slice Units. gitlab # [ 5.850834] systemd[1]: Reached target Swaps. gitlab # [ 5.852231] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 5.854201] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 5.855614] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 5.857200] systemd[1]: Listening on Factory Reset Management. gitlab # [ 5.858160] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 5.860562] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 5.861990] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 5.863930] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 5.865166] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 5.866322] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.867717] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 5.870515] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 5.871544] systemd[1]: Listening on udev Control Socket. gitlab # [ 5.872407] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 5.875116] systemd[1]: Mounting Huge Pages File System... gitlab # [ 5.877214] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 5.879257] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 5.882767] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 5.889700] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 5.890801] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 5.894886] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 5.895807] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 5.897455] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 5.912168] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 5.914941] systemd[1]: Mounting FUSE Control File System... gitlab # [ 5.915868] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 5.922607] systemd[1]: Starting Journal Service... gitlab # [ 5.930698] systemd[1]: Starting Load Kernel Modules... gitlab # [ 5.937257] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 5.941581] systemd-journald[321]: Collecting audit messages is enabled. gitlab # [ 5.943648] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 5.945674] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.952658] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 5.957607] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 5.958586] systemd[1]: Mounted Huge Pages File System. gitlab # [ 5.959445] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 5.811715] systemd[1]: Queued start job for default target M[ 5.960410] systemd[1]: Started Journal Service. gitlab # ulti-User System. gitlab # [ 5.813726] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 5.814989] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 5.816473] systemd-modules-load[323]: Using 3 probe threads gitlab # [ 5.817760] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 5.818866] systemd[1]: Mounted Kernel Trace File System.[ 5.967592] EXT4-fs (vda): re-mounted ea38587d-7a99-4321-ac57-295a3bb58a19. gitlab # gitlab # [ 5.820687] systemd[1]: Finished Create List of Static Device Nodes.[ 5.969513] loop: module loaded gitlab # gitlab # [ 5.822277] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 5.823643] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 5.824759] systemd[1]: Mounted FUSE Control File System. gitlab # [ 5.825879] systemd-modules-load[323]: Inserted module 'loop' gitlab # [ 5.826944] systemd-modules-load[323]: Inserted module 'tls' gitlab # [ 5.827869] systemd[1]: Finished Load Kernel Modules. gitlab # [ 5.828666] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 5.829630] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 5.831278] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 5.833964] systemd[1]: Starting Firewall... gitlab # [ 5.836132] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 5.837142] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 5.843659] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 5.847297] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 5.998727] systemd-journald[321]: Received client request to flush runtime journal. gitlab # [ 5.886819] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 5.888399] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.890359] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 5.891531] systemd-oomd[324]: No swap; memory pressure usage will be degraded gitlab # [ 5.893570] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 5.894502] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 5.895523] systemd[1]: Reached target First Boot Complete. gitlab # [ 5.896346] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 5.897094] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 5.898149] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 5.899066] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 5.900174] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 5.901190] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 5.902129] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 5.925596] systemd-udevd[361]: Using default interface naming scheme 'v261'. gitlab # [ 5.957042] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 5.957957] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.024817] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.040539] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 6.065518] (udev-worker)[427]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 6.067588] (udev-worker)[427]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.091506] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 6.093195] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 6.094936] (udev-worker)[433]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.096693] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.098205] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.100334] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.102260] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.103584] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.283968] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 6.288695] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 6.290594] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 6.297525] ACPI: button: Power Button [PWRF] gitlab # [ 6.304195] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 6.310361] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 6.311327] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 6.312592] mousedev: PS/2 mouse device common for all mice gitlab # [ 6.313140] rtc_cmos 00:05: registered as rtc0 gitlab # [ 6.166245] systemd[1]: Finished Firewall. gitlab # [ 6.316643] rtc_cmos 00:05: setting system clock to 2026-11-13T10:33:49 UTC (1794566029) gitlab # [ 6.318162] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 6.320594] systemd-journald[321]: Time jumped backwards, rotating. gitlab # [ 6.328419] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 6.338015] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 6.346257] FDC 0 is a S82078B gitlab # [ 6.351940] Console: switching to colour dummy device 80x25 gitlab # [ 6.353762] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 6.353764] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 6.359285] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 6.360244] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 6.366133] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 6.375911] Console: switching to colour frame buffer device 160x50 gitlab # [ 6.378909] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 6.380179] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 gitlab # [ 6.383763] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6 gitlab # [ 6.293232] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.315869] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.322510] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.340077] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.346717] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.497541] ppdev: user-space parallel port driver gitlab # [ 6.350129] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.354436] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.505451] kvm_amd: TSC scaling supported gitlab # [ 6.505968] kvm_amd: Nested Virtualization enabled gitlab # [ 6.506435] kvm_amd: Nested Paging enabled gitlab # [ 6.507097] kvm_amd: LBR virtualization supported gitlab # [ 6.507776] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 6.508248] kvm_amd: Virtual GIF supported gitlab # [ 6.508992] kvm_amd: Virtual NMI enabled gitlab # [ 6.540583] EDAC MC: Ver: 3.0.0 gitlab # [ 6.425180] systemd-vconsole-setup[497]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 6.426847] systemd[1]: Finished Virtual Console Setup. gitlab # [ 6.686814] systemd[1]: Mounting /run/wrappers... gitlab # [ 6.709278] systemd[1]: Mounted /run/wrappers. gitlab # [ 6.711101] systemd[1]: Reached target Local File Systems. gitlab # [ 6.712923] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 6.715220] systemd[1]: Starting register-nix-paths.service... gitlab # [ 6.717758] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 6.719862] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.722727] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 6.728736] systemd[1]: Starting Create System Files and Directories... gitlab # [ 6.762628] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 6.765869] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 6.784111] systemd[1]: Finished Create System Files and Directories. gitlab # [ 6.788906] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 6.791509] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 6.809309] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 6.822853] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 6.826029] systemd[1]: Starting Update is Completed... gitlab # [ 6.841581] systemd[1]: Finished Update is Completed. gitlab # [ 6.898947] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 6.901684] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.118780] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.121151] systemd[1]: Reached target System Initialization. gitlab # [ 7.123301] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.125881] systemd[1]: Started logrotate.timer. gitlab # [ 7.127741] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.130181] systemd[1]: Reached target Timer Units. gitlab # [ 7.132141] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.134468] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.136455] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.138467] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.141446] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.143548] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.145872] systemd[1]: Reached target Socket Units. gitlab # [ 7.197933] dbus-broker-launch[579]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.201376] dbus-broker-launch[579]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.204159] dbus-broker-launch[579]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.209375] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.230315] dbus-broker-launch[579]: Ready gitlab # [ 7.233361] systemd[1]: Reached target Basic System. gitlab # [ 7.236448] systemd[1]: Started backdoor.service. gitlab # [ 7.239293] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.242226] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.245666] systemd[1]: Starting Logrotate configuration check... gitlab # [ 7.253276] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.257541] systemd[1]: Starting Post-Boot Actions... gitlab # [ 7.262661] systemd[1]: Started Reset console on configuration changes. gitlab # [ 7.265151] logrotate[584]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # [ 7.270153] logrotate[584]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf gitlab # [ 7.275168] logrotate[584]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 7.277152] logrotate[584]: Reading state from file: /var/lib/logrotate.status gitlab # [ 7.280114] logrotate[584]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.282056] logrotate[584]: Allocating hash table for state file, size 64 entries gitlab # connecting to host... gitlab # [ 7.284176] logrotate[584]: Handling 4 logs gitlab # [ 7.285157] logrotate[584]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.286040] logrotate[584]: considering log /var/log/btmp gitlab # [ 7.286925] logrotate[584]: Creating new state gitlab # [ 7.288246] logrotate[584]: Now: 2026-11-13 10:33 gitlab # [ 7.289153] logrotate[584]: Last rotated at 2026-11-13 10:00 gitlab # [ 7.289809] logrotate[584]: log does not need rotating (log has already been rotated) gitlab # [ 7.290438] logrotate[584]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.290972] logrotate[584]: considering log /var/log/wtmp gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 7.291626] logrotate[584]: Creating new state gitlab # [ 7.292263] logrotate[584]: Now: 2026-11-13 10:33 gitlab # [ 7.292743] logrotate[584]: Last rotated at 2026-11-13 10:00 gitlab: connected to guest root shell gitlab # [ 7.293402] logrotate[584]: log does not need rotating (log has already been rotated) gitlab: (connecting took 7.97 seconds) gitlab # [ 7.293865] logrotate[584]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab: (finished: waiting for the VM to finish booting, in 7.97 seconds) gitlab # [ 7.294546] logrotate[584]: switching euid from 0 to 165 and egid from 0 to 165 (pid 584) gitlab # [ 7.295032] logrotate[584]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.295627] logrotate[584]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.296139] logrotate[584]: Creating new state gitlab # [ 7.296533] logrotate[584]: switching euid from 165 to 0 and egid from 165 to 0 (pid 584) gitlab # [ 7.296980] logrotate[584]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.297582] logrotate[584]: switching euid from 0 to 60 and egid from 0 to 60 (pid 584) gitlab # [ 7.298030] logrotate[584]: considering log /var/log/nginx/*.log gitlab # [ 7.298488] logrotate[584]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.298924] logrotate[584]: Creating new state gitlab # [ 7.299522] logrotate[584]: not running postrotate script, since no logs were rotated gitlab # [ 7.299963] logrotate[584]: switching euid from 60 to 0 and egid from 60 to 0 (pid 584) gitlab # [ 7.300811] systemd[1]: Starting resolvconf update... gitlab # [ 7.311474] systemd[1]: Finished Logrotate configuration check. gitlab # [ 7.313239] systemd[1]: Finished Post-Boot Actions. gitlab # [ 7.319257] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 7.332105] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.333643] nsncd[585]: Nov 13 10:33:50.664 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.334029] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.334628] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.339446] systemd[1]: Starting User Login Management... gitlab # [ 7.364662] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.364856] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.365682] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.367461] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.367783] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 7.369904] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 7.371519] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 7.371810] systemd[1]: nscd.service: Consumed 37ms CPU time over 118ms wall clock time, 7.3M memory peak, 5.5M read from disk. gitlab # [ 7.376842] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.380339] systemd-logind[637]: New seat seat0. gitlab # [ 7.384292] systemd-logind[637]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 7.384738] systemd-logind[637]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard) gitlab # [ 7.385498] systemd-logind[637]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 7.387695] systemd[1]: Started User Login Management. gitlab # [ 7.392050] systemd[1]: Starting linger-users.service... gitlab # [ 7.395621] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.396220] nsncd[675]: Nov 13 10:33:50.727 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.396800] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.397472] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.409128] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 7.409596] systemd[1]: Finished linger-users.service. gitlab # [ 7.413948] systemd[1]: Finished resolvconf update. gitlab # [ 7.414877] systemd[1]: Reached target Preparation for Network. gitlab # [ 7.416941] systemd[1]: Starting DHCP Client... gitlab # [ 7.421175] systemd[1]: Starting Address configuration of eth1... gitlab # [ 7.427396] systemd[1]: Starting Extra networking commands.... gitlab # [ 7.451941] network-addresses-eth1-start[702]: adding address 192.168.1.1/24... done gitlab # [ 7.455743] network-addresses-eth1-start[702]: adding address 2001:db8:1::1/64... done gitlab # [ 7.459848] systemd[1]: Finished Address configuration of eth1. gitlab # [ 7.487045] systemd[1]: Finished Extra networking commands.. gitlab # [ 7.487471] systemd[1]: Reached target Network. gitlab # [ 7.491277] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 7.491788] dhcpcd[710]: dhcpcd-10.3.2 starting gitlab # [ 7.494083] systemd[1]: Starting Nginx Web Server... gitlab # [ 7.494567] dhcpcd[769]: dev: loaded udev gitlab # [ 7.495685] systemd[1]: Starting PostgreSQL Server... gitlab # [ 7.498047] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 7.648820] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 7.500672] s[ 7.649346] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # ystemd[1]: Starting Permit User Sessions... gitlab # [ 7.515817] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 7.519136] systemd[1]: Finished Permit User Sessions. gitlab # [ 7.522443] systemd[1]: Started Getty on tty1. gitlab # [ 7.522783] systemd[1]: Reached target Login Prompts. gitlab # [ 7.744449] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 7.748685] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 7.749450] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 7.750974] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 7.752051] cfg80211: failed to load regulatory.db gitlab # [ 7.619677] nginx-pre-start[791]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok gitlab # [ 7.621966] nginx-pre-start[791]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful gitlab # [ 7.626030] systemd[1]: Started Nginx Web Server. gitlab # [ 7.781256] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 7.635751] redis-server[793]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 7.638519] redis-server[793]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 7.640139] postgresql-pre-start[790]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 7.641932] postgresql-pre-start[790]: This user must also own the server process. gitlab # [ 7.643323] redis-server[793]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=793, just started gitlab # [ 7.645182] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 7.646160] postgresql-pre-start[790]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 7.647592] postgresql-pre-start[790]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 7.649037] postgresql-pre-start[790]: The default text search configuration will be set to "english". gitlab # [ 7.650466] postgresql-pre-start[790]: Data page checksums are enabled. gitlab # [ 7.651547] postgresql-pre-start[790]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 7.652871] postgresql-pre-start[790]: creating subdirectories ... ok gitlab # [ 7.653815] postgresql-pre-start[790]: selecting dynamic shared memory implementation ... posix gitlab # [ 7.654980] redis-server[793]: Configuration loaded gitlab # [ 7.655875] redis[793]: monotonic clock: POSIX clock_gettime gitlab # [ 7.656997] dhcpcd[769]: eth0: waiting for carrier gitlab # [ 7.657894] dhcpcd[769]: eth0: carrier acquired gitlab # [ 7.658313] redis[793]: Running mode=standalone, port=0. gitlab # [ 7.658745] redis[793]: Server initialized gitlab # [ 7.659227] redis[793]: Ready to accept connections unix gitlab # [ 7.659694] redis[793]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 7.663080] dhcpcd[769]: DUID 00:01:00:01:32:89:a4:0e:52:54:00:12:34:56 gitlab # [ 7.664229] dhcpcd[769]: eth0: IAID 00:12:34:56 gitlab # [ 7.665109] dhcpcd[769]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 7.681424] postgresql-pre-start[790]: selecting default "max_connections" ... 100 gitlab # [ 7.724619] postgresql-pre-start[790]: selecting default "shared_buffers" ... 128MB gitlab # [ 7.998877] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.109149] postgresql-pre-start[790]: selecting default time zone ... UTC gitlab # [ 8.111417] postgresql-pre-start[790]: creating configuration files ... ok gitlab # [ 8.177502] gitlab-workhorse[827]: time="2026-11-13T10:33:51Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 8.178320] gitlab-workhorse[827]: time="2026-11-13T10:33:51Z" level=info msg="Using redis/go-redis" gitlab # [ 8.179102] gitlab-workhorse[827]: time="2026-11-13T10:33:51Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 8.264789] postgresql-pre-start[790]: running bootstrap script ... ok gitlab # [ 8.522525] dhcpcd[769]: eth0: soliciting a DHCP lease gitlab # [ 8.676468] NET: Registered PF_PACKET protocol family gitlab # [ 8.536813] dhcpcd[769]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 8.543437] dhcpcd[769]: eth0: probing address 10.0.2.15/24 gitlab # [ 8.694365] postgresql-pre-start[790]: performing post-bootstrap initialization ... ok gitlab # [ 8.932779] postgresql-pre-start[790]: syncing data to disk ... ok gitlab # [ 8.933337] postgresql-pre-start[790]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 8.934056] postgresql-pre-start[790]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 8.934702] postgresql-pre-start[790]: Success. You can now start the database server using: gitlab # [ 8.935528] postgresql-pre-start[790]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 9.058987] postgres[853]: [853] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 9.059798] postgres[853]: [853] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 9.060821] postgres[853]: [853] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 9.064094] postgres[853]: [853] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 9.078223] postgres[862]: [862] LOG: database system was shut down at 2026-11-13 10:33:52 GMT gitlab # [ 9.087608] postgres[853]: [853] LOG: database system is ready to accept connections gitlab # [ 9.093065] systemd[1]: Started PostgreSQL Server. gitlab # [ 9.100469] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 9.284597] postgresql-setup-start[877]: CREATE DATABASE gitlab # [ 9.308964] postgresql-setup-start[883]: CREATE ROLE gitlab # [ 9.318610] postgresql-setup-start[885]: ALTER DATABASE gitlab # [ 9.322221] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[872]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 9.324490] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[872]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 9.344875] postgresql-setup-start[896]: CREATE ROLE gitlab # [ 9.346583] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 9.347132] systemd[1]: Reached target PostgreSQL. gitlab # [ 9.348502] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 9.361945] systemd[1]: Finished gitlab-config.service. gitlab # [ 9.365940] systemd[1]: Started gitaly.service. gitlab # [ 9.368535] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 9.441217] gitlab-postgresql-start[913]: CREATE DATABASE gitlab # [ 9.469232] gitlab-postgresql-start[928]: CREATE EXTENSION gitlab # [ 9.501785] dhcpcd[769]: eth0: soliciting an IPv6 router gitlab # [ 9.506045] dhcpcd[769]: eth0: Router Advertisement from fe80::2 gitlab # [ 9.507045] dhcpcd[769]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 9.507973] dhcpcd[769]: eth0: adding route to fec0::/64 gitlab # [ 9.508799] dhcpcd[769]: eth0: adding default route via fe80::2 gitlab # [ 9.547331] gitlab-postgresql-start[930]: CREATE EXTENSION gitlab # [ 9.550925] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 9.809312] gitaly[906]: time="2026-11-13T10:33:53.138Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=906 gitlab # [ 9.810156] gitaly[906]: time="2026-11-13T10:33:53.138Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=906 gitlab # [ 9.810981] gitaly[906]: time="2026-11-13T10:33:53.138Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=906 gitlab # [ 9.811849] gitaly[906]: time="2026-11-13T10:33:53.138Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=906 gitlab # [ 9.812730] gitaly[906]: time="2026-11-13T10:33:53.138Z" level=info msg="Starting Gitaly" pid=906 version= gitlab # [ 9.987574] gitaly[906]: time="2026-11-13T10:33:53.319Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=180 pid=906 gitlab # [ 9.989415] gitaly[906]: time="2026-11-13T10:33:53.319Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=906 gitlab # [ 9.991205] gitaly[906]: time="2026-11-13T10:33:53.319Z" level=info msg="finished initializing command factory" duration_ms=0 pid=906 gitlab # [ 9.992898] gitaly[906]: time="2026-11-13T10:33:53.319Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-906/git-exec-1649012793.d/git pid=906 gitlab # [ 9.995164] gitaly[906]: time="2026-11-13T10:33:53.323Z" level=info msg="finished detecting git version" duration_ms=3 pid=906 gitlab # [ 9.996994] gitaly[906]: time="2026-11-13T10:33:53.323Z" level=info msg="using Git version" pid=906 version=2.53.g gitlab # [ 9.998604] gitaly[906]: time="2026-11-13T10:33:53.324Z" level=info msg="clearing disk cache object folder" pid=906 storage=default gitlab # [ 10.000404] gitaly[906]: time="2026-11-13T10:33:53.324Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1738298792 pid=906 storage=default gitlab # [ 10.003134] gitaly[906]: time="2026-11-13T10:33:53.324Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1738298792 pid=906 storage=default gitlab # [ 10.006199] gitaly[906]: time="2026-11-13T10:33:53.324Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=906 gitlab # [ 10.009023] gitaly[906]: time="2026-11-13T10:33:53.324Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=906 gitlab # [ 10.011851] gitaly[906]: time="2026-11-13T10:33:53.324Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1738298792 clear_duration_ms=0 pid=906 storage=default gitlab # [ 10.026506] gitaly[906]: time="2026-11-13T10:33:53.358Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=906 gitlab # [ 10.028970] gitaly[906]: time="2026-11-13T10:33:53.358Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=906 schema=unix gitlab # [ 10.031199] gitaly[906]: time="2026-11-13T10:33:53.358Z" level=info msg="finished tempdir cleaner walk" pid=906 storage=default time_ms=0 gitlab # [ 10.032969] gitaly[906]: time="2026-11-13T10:33:53.358Z" level=info msg="listening at address" address=/run/gitaly/gitaly-906/sock.d/intern pid=906 schema=unix gitlab # [ 10.035044] gitaly[906]: time="2026-11-13T10:33:53.358Z" level=info msg="counting repositories" pid=906 gitlab # [ 10.036472] gitaly[906]: time="2026-11-13T10:33:53.358Z" level=info msg="starting to count repositories" pid=906 storage_path=/var/gitlab/state/repositories gitlab # [ 10.038640] gitaly[906]: time="2026-11-13T10:33:53.358Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=906 storage_path=/var/gitlab/state/repositories gitlab # [ 10.041102] gitaly[906]: time="2026-11-13T10:33:53.358Z" level=info msg="completed counting all repositories" counting_duration=0 pid=906 gitlab # [ 10.043960] gitaly[906]: time="2026-11-13T10:33:53.375Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=906 gitlab # [ 10.044988] gitaly[906]: time="2026-11-13T10:33:53.375Z" level=info msg="Started Gitaly" duration_ms=237 pid=906 gitlab # [ 10.045424] gitaly[906]: time="2026-11-13T10:33:53.375Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=906 gitlab # [ 10.045927] gitaly[906]: time="2026-11-13T10:33:53.376Z" level=info msg="maintenance: daily scheduled" pid=906 scheduled="2026-11-13 12:00:00 +0000 UTC" gitlab # [ 10.829889] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[915]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 10.830696] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[915]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 10.855394] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[915]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.949228] gitaly[906]: time="2026-11-13T10:33:55.280Z" level=info msg="License database preloaded" duration_ms=2141 pid=906 gitlab # [ 13.760685] dhcpcd[769]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 13.761611] dhcpcd[769]: eth0: adding route to 10.0.2.0/24 gitlab # [ 13.762978] dhcpcd[769]: eth0: adding default route via 10.0.2.2 gitlab # [ 13.880421] systemd[1]: Started DHCP Client. gitlab # [ 13.882740] systemd[1]: Reached target Network is Online. gitlab # [ 39.547748] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[915]: W, [2026-11-13T10:34:22.878771 #915] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 41.397674] postgres[1302]: [1302] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 41.398106] postgres[1302]: [1302] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 41.399600] postgres[1302]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 41.401476] postgres[1302]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 41.403059] postgres[1302]: attidentity AS identity, gitlab # [ 41.404993] postgres[1302]: attgenerated as attgenerated gitlab # [ 41.406564] postgres[1302]: FROM pg_attribute a gitlab # [ 41.408505] postgres[1302]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 41.410423] postgres[1302]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 41.411868] postgres[1302]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 41.414153] postgres[1302]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 41.416106] postgres[1302]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 41.417928] postgres[1302]: ORDER BY a.attnum gitlab # [ 41.419883] postgres[1302]: gitlab # [ 44.591568] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1335]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 44.592061] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1335]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 48.237577] gitlab-workhorse[827]: time="2026-11-13T10:34:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 58.247250] gitlab-workhorse[827]: time="2026-11-13T10:34:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 59.517580] postgres[1401]: [1401] ERROR: relation "table_without_models" does not exist gitlab # [ 59.517890] postgres[1401]: [1401] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 68.257964] gitlab-workhorse[827]: time="2026-11-13T10:34:51Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 70.821511] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 70.826466] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 70.830699] systemd[1]: Starting gitlab.service... gitlab # [ 71.188579] bundle[1568]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 71.189149] bundle[1568]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 71.190728] bundle[1568]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 73.775208] sidekiq-cluster[1567]: {"severity":"INFO","time":"2026-11-13T10:34:57.090Z","message":"Starting cluster with 1 processes"} gitlab # [ 73.949270] bundle[1568]: {"timestamp":"2026-11-13T10:34:57.257Z","pid":1568,"message":"* Enabling systemd notification integration"} gitlab # [ 73.949936] bundle[1568]: {"timestamp":"2026-11-13T10:34:57.280Z","pid":1568,"message":"Puma starting in cluster mode..."} gitlab # [ 73.950645] bundle[1568]: {"timestamp":"2026-11-13T10:34:57.280Z","pid":1568,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 73.951499] bundle[1568]: {"timestamp":"2026-11-13T10:34:57.280Z","pid":1568,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 73.952308] bundle[1568]: {"timestamp":"2026-11-13T10:34:57.280Z","pid":1568,"message":"* Min threads: 0"} gitlab # [ 73.952935] bundle[1568]: {"timestamp":"2026-11-13T10:34:57.280Z","pid":1568,"message":"* Max threads: 4"} gitlab # [ 73.953629] bundle[1568]: {"timestamp":"2026-11-13T10:34:57.280Z","pid":1568,"message":"* Environment: production"} gitlab # [ 73.954339] bundle[1568]: {"timestamp":"2026-11-13T10:34:57.280Z","pid":1568,"message":"* Master PID: 1568"} gitlab # [ 73.954954] bundle[1568]: {"timestamp":"2026-11-13T10:34:57.280Z","pid":1568,"message":"* Workers: 2"} gitlab # [ 73.955618] bundle[1568]: {"timestamp":"2026-11-13T10:34:57.280Z","pid":1568,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 73.956301] bundle[1568]: {"timestamp":"2026-11-13T10:34:57.280Z","pid":1568,"message":"* Preloading application"} gitlab # [ 78.268069] gitlab-workhorse[827]: time="2026-11-13T10:35:01Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 88.278088] gitlab-workhorse[827]: time="2026-11-13T10:35:11Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 96.180454] bundle[1568]: W, [2026-11-13T10:35:19.511718 #1568] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 97.982614] sidekiq-cluster[1598]: 2026-11-13T10:35:21.313Z pid=1598 tid=wie INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1598, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>449150976} gitlab # [ 98.143543] sidekiq-cluster[1598]: W, [2026-11-13T10:35:21.475195 #1598] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 98.183607] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:21.515Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 98.287788] gitlab-workhorse[827]: time="2026-11-13T10:35:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 100.088183] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.419Z","message":"GitLab reliable fetch activated!"} gitlab # [ 100.092116] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.419Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 100.108653] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.440Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 100.114646] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.442Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 100.114751] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.445Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 100.126790] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.458Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 100.137564] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.469Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 100.147135] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.478Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 100.151581] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.483Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 100.154563] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.486Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 100.165026] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.496Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 100.168888] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.500Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 100.178238] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.509Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 100.186395] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.518Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 100.189814] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.521Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 100.197200] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.528Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 100.203816] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.535Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 100.209930] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.541Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 100.215696] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.547Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 100.221130] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.552Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 100.226331] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.558Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 100.231542] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.563Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 100.236962] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.568Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 100.241967] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.573Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 100.246731] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.578Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 100.251375] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.583Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 100.255554] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.587Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 100.259914] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.591Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 100.264081] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.595Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 100.268504] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.600Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 100.273124] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.604Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 100.278106] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.609Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 100.282297] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.614Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 100.287112] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.618Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 100.293554] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.625Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 100.297794] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.629Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 100.301045] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.632Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 100.303544] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.635Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 100.306401] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.638Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 100.309831] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.641Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 100.312757] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.644Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 100.315369] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.647Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 100.317542] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.649Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 100.319542] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.651Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 100.322207] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.653Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 100.324339] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.656Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 100.326581] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.658Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 100.328976] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.660Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 100.331252] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.663Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 100.333735] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.665Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 100.336339] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.668Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 100.338195] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.669Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 100.340784] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.672Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 100.365830] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.696Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 100.374580] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.706Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 100.377656] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.709Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 100.380206] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.711Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 100.382222] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.714Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 100.384851] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.716Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 100.386980] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.718Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 100.389384] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.721Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 100.391344] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.723Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 100.394105] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.725Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 100.395714] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.727Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 100.398685] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.730Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 100.401303] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.733Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 100.403224] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.735Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 100.406243] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.738Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 100.407872] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.739Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 100.410421] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.742Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 100.412582] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.744Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 100.415317] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.747Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 100.417369] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.749Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 100.419281] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.751Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 100.421428] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.753Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 100.423286] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.755Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 100.425560] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.757Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 100.427980] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.759Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 100.430551] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.762Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 100.432592] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.764Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 100.434898] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.766Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 100.436735] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.768Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 100.438667] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.770Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 100.441314] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.773Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 100.444154] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.775Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 100.446133] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.777Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 100.448144] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.779Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 100.450356] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.782Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 100.452573] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.784Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 100.454895] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.786Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 100.456804] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.788Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 100.459294] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.791Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 100.460850] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.792Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 100.463243] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.795Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 100.464940] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.796Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 100.466846] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.798Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 100.469625] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.801Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 100.471788] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.803Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 100.473362] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.805Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 100.475272] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.807Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 100.482097] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.813Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 100.483730] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.815Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 100.485753] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.817Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 100.488403] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.820Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 100.490386] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.822Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 100.492428] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.824Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 100.494345] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.826Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 100.495942] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.827Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 100.497869] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.829Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 100.500490] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.832Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 100.502210] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:23.834Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 103.698355] sidekiq-cluster[1598]: InitializerConnections Query: SET client_min_messages TO 'warning' gitlab # [ 103.699242] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 103.699993] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 103.700774] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 103.705238] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 103.705909] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 103.707047] sidekiq-cluster[1598]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 103.712403] sidekiq-cluster[1598]: InitializerConnections Query: SET standard_conforming_strings = on gitlab # [ 103.716773] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 103.718252] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 103.719131] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 103.720041] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 103.720937] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 103.722216] sidekiq-cluster[1598]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 103.723356] sidekiq-cluster[1598]: InitializerConnections Query: SET intervalstyle = iso_8601 gitlab # [ 103.724357] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 103.725259] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 103.726187] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 103.727075] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 103.727971] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 103.729228] sidekiq-cluster[1598]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 103.730120] sidekiq-cluster[1598]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname gitlab # [ 103.730670] sidekiq-cluster[1598]: FROM pg_type as t gitlab # [ 103.731587] sidekiq-cluster[1598]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz') gitlab # [ 103.732764] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 103.733662] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 103.734563] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 103.735514] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 103.735968] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 103.736587] sidekiq-cluster[1598]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 103.737053] sidekiq-cluster[1598]: InitializerConnections Query: SET SESSION timezone TO 'UTC' gitlab # [ 103.737525] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 103.737967] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 103.738583] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 103.739063] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 103.739516] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 103.739967] sidekiq-cluster[1598]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 103.740609] sidekiq-cluster[1598]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'per_thread_db_connection_pool_metrics' gitlab # [ 103.741061] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 103.741534] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 103.741984] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 103.742600] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 103.743069] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 103.743524] sidekiq-cluster[1598]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 108.288585] gitlab-workhorse[827]: time="2026-11-13T10:35:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 118.298292] gitlab-workhorse[827]: time="2026-11-13T10:35:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 124.407804] postgres[2027]: [2027] ERROR: relation "table_without_models" does not exist gitlab # [ 124.408415] postgres[2027]: [2027] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 125.435773] postgres[2038]: [2038] ERROR: relation "table_without_models" does not exist gitlab # [ 125.436203] postgres[2038]: [2038] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 126.892982] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:50.224Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 126.898053] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:50.229Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 126.898317] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:50.229Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 126.898742] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:50.229Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 126.903861] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:50.235Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 128.183512] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:51.514Z","message":"Cleaning working queues"} gitlab # [ 128.188849] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:51.520Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"651e2ebabc625fcac7128979","created_at":"2026-11-13T10:34:53.815Z","trace_propagation_headers":{"sentry-trace":"c06cebc263c64927ac9009bf326bb1a8-ff59cde160a34177","baggage":"sentry-trace_id=c06cebc263c64927ac9009bf326bb1a8,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"efac198640975b9c5636644c7fff0557","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-13T10:34:53.828Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-651e2ebabc625fcac7128979: start","job_status":"start","queue_duration_s":57.691267,"scheduling_latency_s":57.691333} gitlab # [ 128.254603] bundle[1568]: {"timestamp":"2026-11-13T10:35:51.585Z","pid":1568,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 128.255387] bundle[1568]: {"timestamp":"2026-11-13T10:35:51.585Z","pid":1568,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 128.256526] bundle[1568]: {"timestamp":"2026-11-13T10:35:51.585Z","pid":1568,"message":"! #\u003cThread:0x00007a37513f78e0 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 128.257887] bundle[1568]: {"timestamp":"2026-11-13T10:35:51.585Z","pid":1568,"message":"! #\u003cThread:0x00007a37516a4548 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 128.259419] bundle[1568]: {"timestamp":"2026-11-13T10:35:51.585Z","pid":1568,"message":"Use Ctrl-C to stop"} gitlab # [ 128.260526] bundle[1568]: {"timestamp":"2026-11-13T10:35:51.586Z","pid":1568,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 128.402873] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:35:51.734Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"651e2ebabc625fcac7128979","created_at":"2026-11-13T10:34:53.815Z","trace_propagation_headers":{"sentry-trace":"c06cebc263c64927ac9009bf326bb1a8-ff59cde160a34177","baggage":"sentry-trace_id=c06cebc263c64927ac9009bf326bb1a8,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"efac198640975b9c5636644c7fff0557","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-13T10:34:53.828Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-651e2ebabc625fcac7128979: done: 0.214238 sec","job_status":"done","queue_duration_s":57.691267,"scheduling_latency_s":57.691333,"redis_calls":26,"redis_duration_s":0.033644,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.003387,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001918,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.028339,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.086256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.100736266,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.214238,"completed_at":"2026-11-13T10:35:51.734Z","load_balancing_strategy":"primary","db_duration_s":0.044595,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 128.576959] bundle[1568]: {"timestamp":"2026-11-13T10:35:51.908Z","pid":1568,"message":"- Worker 0 (PID: 2136) booted in 0.27s, phase: 0"} gitlab # [ 128.596772] bundle[1568]: {"timestamp":"2026-11-13T10:35:51.928Z","pid":1568,"message":"- Worker 1 (PID: 2141) booted in 0.27s, phase: 0"} gitlab # [ 128.603540] systemd[1]: Started gitlab.service. gitlab # [ 128.605335] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 128.608897] systemd[1]: Starting setup-gitlab.service... gitlab # [ 128.628726] setup-gitlab-start[2169]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 130.874441] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:35:54 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 2223 gitlab # [ 130.880236] setup-gitlab-start[2169]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 130.880957] setup-gitlab-start[2169]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 130.904784] runuser[2189]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 131.289891] setup-gitlab-start[2190]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 131.290632] setup-gitlab-start[2190]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 131.292292] setup-gitlab-start[2190]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 131.858194] setup-gitlab-start[2190]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 131.858600] setup-gitlab-start[2190]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 131.860172] setup-gitlab-start[2190]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 139.918700] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:03.249Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f4b88acfa0ea018c2bf2852a","created_at":"2026-11-13T10:36:03.232Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.248Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f4b88acfa0ea018c2bf2852a: start","job_status":"start","queue_duration_s":0.001418,"scheduling_latency_s":0.00144} gitlab # [ 140.241911] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:03.573Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f4b88acfa0ea018c2bf2852a","created_at":"2026-11-13T10:36:03.232Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.248Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f4b88acfa0ea018c2bf2852a: done: 0.323738 sec","job_status":"done","queue_duration_s":0.001418,"scheduling_latency_s":0.00144,"redis_calls":71,"redis_duration_s":0.124224,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.018945,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.018354,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.083272,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.003653,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.092787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00176922,"duration_s":0.323738,"completed_at":"2026-11-13T10:36:03.573Z","load_balancing_strategy":"primary","db_duration_s":0.018957,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 140.299288] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:03.630Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"dd1702453aa7786f373b8caa","created_at":"2026-11-13T10:36:03.287Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ddc8f39a75daae8bb98eea18ffe517b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.299Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-dd1702453aa7786f373b8caa: start","job_status":"start","queue_duration_s":0.331591,"scheduling_latency_s":0.331615} gitlab # [ 140.377959] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:03.709Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"dd1702453aa7786f373b8caa","created_at":"2026-11-13T10:36:03.287Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ddc8f39a75daae8bb98eea18ffe517b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.299Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-dd1702453aa7786f373b8caa: done: 0.078574 sec","job_status":"done","queue_duration_s":0.331591,"scheduling_latency_s":0.331615,"redis_calls":19,"redis_duration_s":0.027238000000000002,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001855,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000496,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.022665,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002222,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.015978008,"duration_s":0.078574,"completed_at":"2026-11-13T10:36:03.709Z","load_balancing_strategy":"primary","db_duration_s":0.005165,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 140.429814] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:03.760Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"bdc929c9d5b6f77a792df2e3","created_at":"2026-11-13T10:36:03.318Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb6db22d8f96102f108aad702cb625fc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-13T10:36:03.329Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-bdc929c9d5b6f77a792df2e3: start","job_status":"start","queue_duration_s":0.431172,"scheduling_latency_s":0.43121} gitlab # [ 140.447103] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:03.778Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 140.465203] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:03.796Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"bdc929c9d5b6f77a792df2e3","created_at":"2026-11-13T10:36:03.318Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb6db22d8f96102f108aad702cb625fc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-13T10:36:03.329Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-bdc929c9d5b6f77a792df2e3: done: 0.035562 sec","job_status":"done","queue_duration_s":0.431172,"scheduling_latency_s":0.43121,"redis_calls":10,"redis_duration_s":0.005342,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000974,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003494,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000874,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000864076,"duration_s":0.035562,"completed_at":"2026-11-13T10:36:03.796Z","load_balancing_strategy":"primary","db_duration_s":0.001683,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 140.504627] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:03.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"aeb91fb6f91be2e5fd412277","created_at":"2026-11-13T10:36:03.336Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d777e9e8b419c03ecad77fc03aac739","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.342Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-aeb91fb6f91be2e5fd412277: start","job_status":"start","queue_duration_s":0.49367,"scheduling_latency_s":0.493763} gitlab # [ 140.768297] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.099Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"aeb91fb6f91be2e5fd412277","created_at":"2026-11-13T10:36:03.336Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d777e9e8b419c03ecad77fc03aac739","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.342Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-aeb91fb6f91be2e5fd412277: done: 0.263807 sec","job_status":"done","queue_duration_s":0.49367,"scheduling_latency_s":0.493763,"redis_calls":14,"redis_duration_s":0.006805,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00238,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001801,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002624,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.053,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.133504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001358274,"exclusive_lock_hold_duration_s":0.20973056699999998,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.263807,"completed_at":"2026-11-13T10:36:04.099Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.076575,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 140.839436] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.170Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"b14746f1303440a0db48d5d1","created_at":"2026-11-13T10:36:03.345Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9e611b64fbd10fdb1793044f87efbff","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.349Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-b14746f1303440a0db48d5d1: start","job_status":"start","queue_duration_s":0.821203,"scheduling_latency_s":0.821233} gitlab # [ 141.088867] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.420Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"b14746f1303440a0db48d5d1","created_at":"2026-11-13T10:36:03.345Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9e611b64fbd10fdb1793044f87efbff","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.349Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-b14746f1303440a0db48d5d1: done: 0.249242 sec","job_status":"done","queue_duration_s":0.821203,"scheduling_latency_s":0.821233,"redis_calls":565,"redis_duration_s":0.068939,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000795,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.067898,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.203168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000222375,"duration_s":0.249242,"completed_at":"2026-11-13T10:36:04.420Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000482,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.162538] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.493Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"17861319fd5efd30411e1e00","created_at":"2026-11-13T10:36:03.368Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4783b749038a77f267a1801a7883680c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.376Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-17861319fd5efd30411e1e00: start","job_status":"start","queue_duration_s":1.116949,"scheduling_latency_s":1.117054} gitlab # [ 141.203341] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.534Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"17861319fd5efd30411e1e00","created_at":"2026-11-13T10:36:03.368Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4783b749038a77f267a1801a7883680c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.376Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-17861319fd5efd30411e1e00: done: 0.040809 sec","job_status":"done","queue_duration_s":1.116949,"scheduling_latency_s":1.117054,"redis_calls":12,"redis_duration_s":0.002339,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000791,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00112,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000430223,"duration_s":0.040809,"completed_at":"2026-11-13T10:36:04.534Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000976,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.241813] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.572Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"883797f907f48e299946d200","created_at":"2026-11-13T10:36:03.390Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"04276a3e37e2eece77c01b9e4e08e2a2","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.397Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-883797f907f48e299946d200: start","job_status":"start","queue_duration_s":1.17455,"scheduling_latency_s":1.17457} gitlab # [ 141.277337] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.608Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"883797f907f48e299946d200","created_at":"2026-11-13T10:36:03.390Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"04276a3e37e2eece77c01b9e4e08e2a2","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.397Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-883797f907f48e299946d200: done: 0.036228 sec","job_status":"done","queue_duration_s":1.17455,"scheduling_latency_s":1.17457,"redis_calls":10,"redis_duration_s":0.001571,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000247,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001041,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000283,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000335797,"duration_s":0.036228,"completed_at":"2026-11-13T10:36:04.608Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002857,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.297089] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.610Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"db9c6bf2100130563011aa5c","created_at":"2026-11-13T10:36:03.408Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5135048a27dab8bf6e552a1532e19e7d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-13T10:36:03.415Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-db9c6bf2100130563011aa5c: start","job_status":"start","queue_duration_s":1.194417,"scheduling_latency_s":1.194446} gitlab # [ 141.342342] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.673Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"db9c6bf2100130563011aa5c","created_at":"2026-11-13T10:36:03.408Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5135048a27dab8bf6e552a1532e19e7d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-13T10:36:03.415Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-db9c6bf2100130563011aa5c: done: 0.063466 sec","job_status":"done","queue_duration_s":1.194417,"scheduling_latency_s":1.194446,"redis_calls":10,"redis_duration_s":0.001732,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000393,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001029,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00031,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000250591,"duration_s":0.063466,"completed_at":"2026-11-13T10:36:04.673Z","load_balancing_strategy":"primary","db_duration_s":0.038332,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.362841] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.675Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d9696a2997fe41c9af8c09fd","created_at":"2026-11-13T10:36:03.422Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3cd3ac131564c0a506141dcb675023da","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.430Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d9696a2997fe41c9af8c09fd: start","job_status":"start","queue_duration_s":1.245494,"scheduling_latency_s":1.245534} gitlab # [ 141.387064] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.718Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d9696a2997fe41c9af8c09fd","created_at":"2026-11-13T10:36:03.422Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3cd3ac131564c0a506141dcb675023da","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.430Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d9696a2997fe41c9af8c09fd: done: 0.042732 sec","job_status":"done","queue_duration_s":1.245494,"scheduling_latency_s":1.245534,"redis_calls":16,"redis_duration_s":0.003158,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001689,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001108,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000361,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026848,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000419885,"duration_s":0.042732,"completed_at":"2026-11-13T10:36:04.718Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006098,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.390056] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.720Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"77519f1f30b9e07655828758","created_at":"2026-11-13T10:36:03.451Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57e12cb64944432fc40eaede40adf48a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.461Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-77519f1f30b9e07655828758: start","job_status":"start","queue_duration_s":1.258683,"scheduling_latency_s":1.258708} gitlab # [ 141.428849] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.760Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"77519f1f30b9e07655828758","created_at":"2026-11-13T10:36:03.451Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57e12cb64944432fc40eaede40adf48a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.461Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-77519f1f30b9e07655828758: done: 0.039612 sec","job_status":"done","queue_duration_s":1.258683,"scheduling_latency_s":1.258708,"redis_calls":10,"redis_duration_s":0.00492,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000346,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004154,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00042,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000314565,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.039612,"completed_at":"2026-11-13T10:36:04.760Z","load_balancing_strategy":"primary","db_duration_s":0.007553,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.470781] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.761Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"153fae16f7a9a9a501ccf5bc","created_at":"2026-11-13T10:36:03.469Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17b16871a6d97d0d2964f41e4555b7a5","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.478Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-153fae16f7a9a9a501ccf5bc: start","job_status":"start","queue_duration_s":1.283806,"scheduling_latency_s":1.28383} gitlab # [ 141.471401] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.775Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"153fae16f7a9a9a501ccf5bc","created_at":"2026-11-13T10:36:03.469Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17b16871a6d97d0d2964f41e4555b7a5","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.478Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-153fae16f7a9a9a501ccf5bc: done: 0.013467 sec","job_status":"done","queue_duration_s":1.283806,"scheduling_latency_s":1.28383,"redis_calls":12,"redis_duration_s":0.001545,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000311,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000236,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000699,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000193041,"duration_s":0.013467,"completed_at":"2026-11-13T10:36:04.775Z","load_balancing_strategy":"primary","db_duration_s":0.000249,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.482691] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.776Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c74f34536ffeb6528eefdf40","created_at":"2026-11-13T10:36:03.496Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3350497a425dfd0f32a9860ea65af9c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.504Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c74f34536ffeb6528eefdf40: start","job_status":"start","queue_duration_s":1.271376,"scheduling_latency_s":1.271388} gitlab # [ 141.519564] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.810Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c74f34536ffeb6528eefdf40","created_at":"2026-11-13T10:36:03.496Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3350497a425dfd0f32a9860ea65af9c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.504Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c74f34536ffeb6528eefdf40: done: 0.034748 sec","job_status":"done","queue_duration_s":1.271376,"scheduling_latency_s":1.271388,"redis_calls":14,"redis_duration_s":0.005451,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00046,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004357,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000634,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000292775,"exclusive_lock_hold_duration_s":0.016421082,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.034748,"completed_at":"2026-11-13T10:36:04.810Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008877,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.532751] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.812Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"1181e3f10aa90c0bba885b39","created_at":"2026-11-13T10:36:03.510Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bbf99673db6ffa427ef186258b797956","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.518Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-1181e3f10aa90c0bba885b39: start","job_status":"start","queue_duration_s":1.293977,"scheduling_latency_s":1.293993} gitlab # [ 141.545100] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.834Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"1181e3f10aa90c0bba885b39","created_at":"2026-11-13T10:36:03.510Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bbf99673db6ffa427ef186258b797956","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.518Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-1181e3f10aa90c0bba885b39: done: 0.022645 sec","job_status":"done","queue_duration_s":1.293977,"scheduling_latency_s":1.293993,"redis_calls":12,"redis_duration_s":0.001532,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000267,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00071,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000555,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000328533,"exclusive_lock_hold_duration_s":0.006562566,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.022645,"completed_at":"2026-11-13T10:36:04.834Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.594710] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"7787eb04f0955633317779d8","created_at":"2026-11-13T10:36:03.530Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5150a916b3b891dc358b6a986058b7ad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.535Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-7787eb04f0955633317779d8: start","job_status":"start","queue_duration_s":1.299799,"scheduling_latency_s":1.299813} gitlab # [ 141.595514] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.843Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 141.598403] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.848Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"7787eb04f0955633317779d8","created_at":"2026-11-13T10:36:03.530Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5150a916b3b891dc358b6a986058b7ad","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.535Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-7787eb04f0955633317779d8: done: 0.012397 sec","job_status":"done","queue_duration_s":1.299799,"scheduling_latency_s":1.299813,"redis_calls":10,"redis_duration_s":0.001294,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000288,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000227124,"duration_s":0.012397,"completed_at":"2026-11-13T10:36:04.848Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00023,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.660576] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"318813b4db43a1d5026c1c50","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-318813b4db43a1d5026c1c50: start","job_status":"start","queue_duration_s":1.3099,"scheduling_latency_s":1.309914} gitlab # [ 141.661212] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"318813b4db43a1d5026c1c50","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-318813b4db43a1d5026c1c50: done: 0.016192 sec","job_status":"done","queue_duration_s":1.3099,"scheduling_latency_s":1.309914,"redis_calls":16,"redis_duration_s":0.002256,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000235,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000735,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001286,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000372952,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016192,"completed_at":"2026-11-13T10:36:04.865Z","load_balancing_strategy":"primary","db_duration_s":0.002152,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.707139] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fb8d6ad34a5c4e1b2ebcc84","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fb8d6ad34a5c4e1b2ebcc84: start","job_status":"start","queue_duration_s":1.327097,"scheduling_latency_s":1.32711} gitlab # [ 141.707801] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fb8d6ad34a5c4e1b2ebcc84","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fb8d6ad34a5c4e1b2ebcc84: done: 0.010148 sec","job_status":"done","queue_duration_s":1.327097,"scheduling_latency_s":1.32711,"redis_calls":13,"redis_duration_s":0.0015570000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001056,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010148,"completed_at":"2026-11-13T10:36:04.876Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.723975] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2fff65aa27d4ef59b604a43","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2fff65aa27d4ef59b604a43: start","job_status":"start","queue_duration_s":1.338111,"scheduling_latency_s":1.338124} gitlab # [ 141.734748] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.886Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2fff65aa27d4ef59b604a43","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2fff65aa27d4ef59b604a43: done: 0.009557 sec","job_status":"done","queue_duration_s":1.338111,"scheduling_latency_s":1.338124,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000474,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009557,"completed_at":"2026-11-13T10:36:04.886Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.786894] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f7c99aca9edbb0c14e7e26d","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f7c99aca9edbb0c14e7e26d: start","job_status":"start","queue_duration_s":1.348614,"scheduling_latency_s":1.348627} gitlab # [ 141.787722] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.897Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f7c99aca9edbb0c14e7e26d","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f7c99aca9edbb0c14e7e26d: done: 0.009581 sec","job_status":"done","queue_duration_s":1.348614,"scheduling_latency_s":1.348627,"redis_calls":13,"redis_duration_s":0.001541,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009581,"completed_at":"2026-11-13T10:36:04.897Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.814029] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"666327de61b6f80008ffa10b","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-666327de61b6f80008ffa10b: start","job_status":"start","queue_duration_s":1.359016,"scheduling_latency_s":1.359029} gitlab # [ 141.825174] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"666327de61b6f80008ffa10b","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-666327de61b6f80008ffa10b: done: 0.009523 sec","job_status":"done","queue_duration_s":1.359016,"scheduling_latency_s":1.359029,"redis_calls":13,"redis_duration_s":0.0015600000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000486,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009523,"completed_at":"2026-11-13T10:36:04.907Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.868110] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4a4a9e3238be48681de05f3","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4a4a9e3238be48681de05f3: start","job_status":"start","queue_duration_s":1.369352,"scheduling_latency_s":1.369364} gitlab # [ 141.871504] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4a4a9e3238be48681de05f3","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4a4a9e3238be48681de05f3: done: 0.010109 sec","job_status":"done","queue_duration_s":1.369352,"scheduling_latency_s":1.369364,"redis_calls":13,"redis_duration_s":0.001657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010109,"completed_at":"2026-11-13T10:36:04.918Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.916758] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1ddcc244e733c860f661328","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1ddcc244e733c860f661328: start","job_status":"start","queue_duration_s":1.380434,"scheduling_latency_s":1.380447} gitlab # [ 141.918085] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1ddcc244e733c860f661328","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1ddcc244e733c860f661328: done: 0.009608 sec","job_status":"done","queue_duration_s":1.380434,"scheduling_latency_s":1.380447,"redis_calls":13,"redis_duration_s":0.001506,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009608,"completed_at":"2026-11-13T10:36:04.929Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.954966] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71ac0b40fd7e90ce936a9332","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71ac0b40fd7e90ce936a9332: start","job_status":"start","queue_duration_s":1.390867,"scheduling_latency_s":1.390879} gitlab # [ 142.003163] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71ac0b40fd7e90ce936a9332","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71ac0b40fd7e90ce936a9332: done: 0.009263 sec","job_status":"done","queue_duration_s":1.390867,"scheduling_latency_s":1.390879,"redis_calls":13,"redis_duration_s":0.001588,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000617,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009263,"completed_at":"2026-11-13T10:36:04.939Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.004606] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"075ddd8a574064b80fc769ad","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-075ddd8a574064b80fc769ad: start","job_status":"start","queue_duration_s":1.401123,"scheduling_latency_s":1.401135} gitlab # [ 142.013639] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"075ddd8a574064b80fc769ad","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-075ddd8a574064b80fc769ad: done: 0.009519 sec","job_status":"done","queue_duration_s":1.401123,"scheduling_latency_s":1.401135,"redis_calls":13,"redis_duration_s":0.0015170000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009519,"completed_at":"2026-11-13T10:36:04.949Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.070709] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e83769208c2fea32879af9d5","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e83769208c2fea32879af9d5: start","job_status":"start","queue_duration_s":1.41164,"scheduling_latency_s":1.411652} gitlab # [ 142.071803] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e83769208c2fea32879af9d5","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e83769208c2fea32879af9d5: done: 0.009488 sec","job_status":"done","queue_duration_s":1.41164,"scheduling_latency_s":1.411652,"redis_calls":13,"redis_duration_s":0.001537,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009488,"completed_at":"2026-11-13T10:36:04.960Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.098500] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccfe666424000631cf4a4176","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccfe666424000631cf4a4176: start","job_status":"start","queue_duration_s":1.42209,"scheduling_latency_s":1.422103} gitlab # [ 142.109537] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ccfe666424000631cf4a4176","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ccfe666424000631cf4a4176: done: 0.009272 sec","job_status":"done","queue_duration_s":1.42209,"scheduling_latency_s":1.422103,"redis_calls":13,"redis_duration_s":0.001572,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000486,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009272,"completed_at":"2026-11-13T10:36:04.970Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.173854] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.971Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0904853611890382194c0dca","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0904853611890382194c0dca: start","job_status":"start","queue_duration_s":1.432308,"scheduling_latency_s":1.43232} gitlab # [ 142.174777] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0904853611890382194c0dca","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0904853611890382194c0dca: done: 0.009395 sec","job_status":"done","queue_duration_s":1.432308,"scheduling_latency_s":1.43232,"redis_calls":13,"redis_duration_s":0.0015339999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009395,"completed_at":"2026-11-13T10:36:04.980Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.221804] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a020e061977322721bdc6d79","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a020e061977322721bdc6d79: start","job_status":"start","queue_duration_s":1.442657,"scheduling_latency_s":1.44267} gitlab # [ 142.222456] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a020e061977322721bdc6d79","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a020e061977322721bdc6d79: done: 0.009406 sec","job_status":"done","queue_duration_s":1.442657,"scheduling_latency_s":1.44267,"redis_calls":13,"redis_duration_s":0.001504,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009406,"completed_at":"2026-11-13T10:36:04.991Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.238054] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:04.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"920b191370e22034e565d6c7","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-920b191370e22034e565d6c7: start","job_status":"start","queue_duration_s":1.45287,"scheduling_latency_s":1.452882} gitlab # [ 142.283852] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.001Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"920b191370e22034e565d6c7","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-920b191370e22034e565d6c7: done: 0.009362 sec","job_status":"done","queue_duration_s":1.45287,"scheduling_latency_s":1.452882,"redis_calls":13,"redis_duration_s":0.001683,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009362,"completed_at":"2026-11-13T10:36:05.001Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.285047] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dae627ff88b374565ded06a","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dae627ff88b374565ded06a: start","job_status":"start","queue_duration_s":1.463088,"scheduling_latency_s":1.4631} gitlab # [ 142.327976] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dae627ff88b374565ded06a","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dae627ff88b374565ded06a: done: 0.009673 sec","job_status":"done","queue_duration_s":1.463088,"scheduling_latency_s":1.4631,"redis_calls":13,"redis_duration_s":0.0015660000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009673,"completed_at":"2026-11-13T10:36:05.011Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.333735] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"deec884c401cf84ca76f9ff6","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-deec884c401cf84ca76f9ff6: start","job_status":"start","queue_duration_s":1.473565,"scheduling_latency_s":1.473584} gitlab # [ 142.342534] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.022Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"deec884c401cf84ca76f9ff6","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-deec884c401cf84ca76f9ff6: done: 0.009539 sec","job_status":"done","queue_duration_s":1.473565,"scheduling_latency_s":1.473584,"redis_calls":13,"redis_duration_s":0.001666,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000652,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009539,"completed_at":"2026-11-13T10:36:05.022Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.413133] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7848fecf4d2180f9c5013e68","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7848fecf4d2180f9c5013e68: start","job_status":"start","queue_duration_s":1.484064,"scheduling_latency_s":1.484077} gitlab # [ 142.413902] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7848fecf4d2180f9c5013e68","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7848fecf4d2180f9c5013e68: done: 0.009374 sec","job_status":"done","queue_duration_s":1.484064,"scheduling_latency_s":1.484077,"redis_calls":13,"redis_duration_s":0.001584,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009374,"completed_at":"2026-11-13T10:36:05.032Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.462739] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5348ce31eea9b14ba99876cc","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5348ce31eea9b14ba99876cc: start","job_status":"start","queue_duration_s":1.494262,"scheduling_latency_s":1.494273} gitlab # [ 142.463704] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5348ce31eea9b14ba99876cc","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5348ce31eea9b14ba99876cc: done: 0.009379 sec","job_status":"done","queue_duration_s":1.494262,"scheduling_latency_s":1.494273,"redis_calls":13,"redis_duration_s":0.001491,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009379,"completed_at":"2026-11-13T10:36:05.042Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.470806] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.043Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cffa2315e7b98fe941c965dc","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cffa2315e7b98fe941c965dc: start","job_status":"start","queue_duration_s":1.504441,"scheduling_latency_s":1.504453} gitlab # [ 142.482295] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cffa2315e7b98fe941c965dc","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cffa2315e7b98fe941c965dc: done: 0.009258 sec","job_status":"done","queue_duration_s":1.504441,"scheduling_latency_s":1.504453,"redis_calls":13,"redis_duration_s":0.001444,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00048,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009258,"completed_at":"2026-11-13T10:36:05.052Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.528641] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fd2c0960500b347ca212a72","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fd2c0960500b347ca212a72: start","job_status":"start","queue_duration_s":1.514658,"scheduling_latency_s":1.514671} gitlab # [ 142.529279] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fd2c0960500b347ca212a72","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fd2c0960500b347ca212a72: done: 0.009404 sec","job_status":"done","queue_duration_s":1.514658,"scheduling_latency_s":1.514671,"redis_calls":13,"redis_duration_s":0.0014559999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009404,"completed_at":"2026-11-13T10:36:05.063Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.575129] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.064Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b10bc3a330b39b6ca5a2612","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b10bc3a330b39b6ca5a2612: start","job_status":"start","queue_duration_s":1.525014,"scheduling_latency_s":1.525027} gitlab # [ 142.575741] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b10bc3a330b39b6ca5a2612","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b10bc3a330b39b6ca5a2612: done: 0.009368 sec","job_status":"done","queue_duration_s":1.525014,"scheduling_latency_s":1.525027,"redis_calls":13,"redis_duration_s":0.001568,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006253,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009368,"completed_at":"2026-11-13T10:36:05.073Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.611078] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d6a3dba8442b5fa523feb65","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d6a3dba8442b5fa523feb65: start","job_status":"start","queue_duration_s":1.535323,"scheduling_latency_s":1.535336} gitlab # [ 142.622635] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d6a3dba8442b5fa523feb65","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d6a3dba8442b5fa523feb65: done: 0.009304 sec","job_status":"done","queue_duration_s":1.535323,"scheduling_latency_s":1.535336,"redis_calls":13,"redis_duration_s":0.001497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009304,"completed_at":"2026-11-13T10:36:05.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.669773] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e86c76528fb8be4e229d6a74","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e86c76528fb8be4e229d6a74: start","job_status":"start","queue_duration_s":1.545623,"scheduling_latency_s":1.545635} gitlab # [ 142.670561] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e86c76528fb8be4e229d6a74","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e86c76528fb8be4e229d6a74: done: 0.009408 sec","job_status":"done","queue_duration_s":1.545623,"scheduling_latency_s":1.545635,"redis_calls":13,"redis_duration_s":0.0014650000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009408,"completed_at":"2026-11-13T10:36:05.094Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.729738] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6854f216bff6f6b43302320e","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6854f216bff6f6b43302320e: start","job_status":"start","queue_duration_s":1.555965,"scheduling_latency_s":1.555977} gitlab # [ 142.730603] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6854f216bff6f6b43302320e","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6854f216bff6f6b43302320e: done: 0.009487 sec","job_status":"done","queue_duration_s":1.555965,"scheduling_latency_s":1.555977,"redis_calls":13,"redis_duration_s":0.001536,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006222,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009487,"completed_at":"2026-11-13T10:36:05.104Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.777536] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f62188ea79363117af0bfef4","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f62188ea79363117af0bfef4: start","job_status":"start","queue_duration_s":1.566499,"scheduling_latency_s":1.566513} gitlab # [ 142.778149] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.116Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f62188ea79363117af0bfef4","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f62188ea79363117af0bfef4: done: 0.010405 sec","job_status":"done","queue_duration_s":1.566499,"scheduling_latency_s":1.566513,"redis_calls":13,"redis_duration_s":0.0019030000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000591,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001312,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010405,"completed_at":"2026-11-13T10:36:05.115Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.797756] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d557a0ce513889807205867","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d557a0ce513889807205867: start","job_status":"start","queue_duration_s":1.578038,"scheduling_latency_s":1.57805} gitlab # [ 142.842751] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d557a0ce513889807205867","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d557a0ce513889807205867: done: 0.009733 sec","job_status":"done","queue_duration_s":1.578038,"scheduling_latency_s":1.57805,"redis_calls":13,"redis_duration_s":0.00164,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000475,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001165,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009733,"completed_at":"2026-11-13T10:36:05.127Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.843958] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a19a5579751671461dc5274","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a19a5579751671461dc5274: start","job_status":"start","queue_duration_s":1.588725,"scheduling_latency_s":1.588738} gitlab # [ 142.891402] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a19a5579751671461dc5274","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a19a5579751671461dc5274: done: 0.009549 sec","job_status":"done","queue_duration_s":1.588725,"scheduling_latency_s":1.588738,"redis_calls":13,"redis_duration_s":0.0015109999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009549,"completed_at":"2026-11-13T10:36:05.137Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.892608] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25a373e769c9c01344f06bc0","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25a373e769c9c01344f06bc0: start","job_status":"start","queue_duration_s":1.599113,"scheduling_latency_s":1.599125} gitlab # [ 142.933865] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25a373e769c9c01344f06bc0","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25a373e769c9c01344f06bc0: done: 0.009451 sec","job_status":"done","queue_duration_s":1.599113,"scheduling_latency_s":1.599125,"redis_calls":13,"redis_duration_s":0.001522,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009451,"completed_at":"2026-11-13T10:36:05.147Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.936889] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50e2d1f3631d96ccb47ee295","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50e2d1f3631d96ccb47ee295: start","job_status":"start","queue_duration_s":1.609373,"scheduling_latency_s":1.609386} gitlab # [ 142.948530] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50e2d1f3631d96ccb47ee295","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50e2d1f3631d96ccb47ee295: done: 0.009265 sec","job_status":"done","queue_duration_s":1.609373,"scheduling_latency_s":1.609386,"redis_calls":13,"redis_duration_s":0.0015119999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009265,"completed_at":"2026-11-13T10:36:05.157Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.989929] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.158Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff86387e8e90c87a17e69ccd","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff86387e8e90c87a17e69ccd: start","job_status":"start","queue_duration_s":1.619707,"scheduling_latency_s":1.619719} gitlab # [ 142.995357] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff86387e8e90c87a17e69ccd","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff86387e8e90c87a17e69ccd: done: 0.00936 sec","job_status":"done","queue_duration_s":1.619707,"scheduling_latency_s":1.619719,"redis_calls":13,"redis_duration_s":0.0015040000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00936,"completed_at":"2026-11-13T10:36:05.168Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.032464] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb3ab74a0fffe5107b2bbbce","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb3ab74a0fffe5107b2bbbce: start","job_status":"start","queue_duration_s":1.630061,"scheduling_latency_s":1.630073} gitlab # [ 143.043903] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.178Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb3ab74a0fffe5107b2bbbce","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb3ab74a0fffe5107b2bbbce: done: 0.009463 sec","job_status":"done","queue_duration_s":1.630061,"scheduling_latency_s":1.630073,"redis_calls":13,"redis_duration_s":0.001532,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001005,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009463,"completed_at":"2026-11-13T10:36:05.178Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.086847] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.179Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee206765bf8a84876b398e36","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee206765bf8a84876b398e36: start","job_status":"start","queue_duration_s":1.640465,"scheduling_latency_s":1.640477} gitlab # [ 143.090972] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee206765bf8a84876b398e36","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee206765bf8a84876b398e36: done: 0.009401 sec","job_status":"done","queue_duration_s":1.640465,"scheduling_latency_s":1.640477,"redis_calls":13,"redis_duration_s":0.001531,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009401,"completed_at":"2026-11-13T10:36:05.188Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.127562] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"223f19ccee3616827843a8b8","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-223f19ccee3616827843a8b8: start","job_status":"start","queue_duration_s":1.650813,"scheduling_latency_s":1.650825} gitlab # [ 143.138192] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"223f19ccee3616827843a8b8","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-223f19ccee3616827843a8b8: done: 0.009644 sec","job_status":"done","queue_duration_s":1.650813,"scheduling_latency_s":1.650825,"redis_calls":13,"redis_duration_s":0.001652,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000485,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001167,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006173,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009644,"completed_at":"2026-11-13T10:36:05.199Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.188621] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47bc866ae94987ac9bb923de","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47bc866ae94987ac9bb923de: start","job_status":"start","queue_duration_s":1.661406,"scheduling_latency_s":1.661418} gitlab # [ 143.189581] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47bc866ae94987ac9bb923de","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47bc866ae94987ac9bb923de: done: 0.009252 sec","job_status":"done","queue_duration_s":1.661406,"scheduling_latency_s":1.661418,"redis_calls":13,"redis_duration_s":0.001491,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009252,"completed_at":"2026-11-13T10:36:05.209Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.217704] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96081e0d34736a5e66a15501","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96081e0d34736a5e66a15501: start","job_status":"start","queue_duration_s":1.671486,"scheduling_latency_s":1.671498} gitlab # [ 143.228656] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"96081e0d34736a5e66a15501","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-96081e0d34736a5e66a15501: done: 0.009632 sec","job_status":"done","queue_duration_s":1.671486,"scheduling_latency_s":1.671498,"redis_calls":13,"redis_duration_s":0.001521,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009632,"completed_at":"2026-11-13T10:36:05.220Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.269540] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1353bae1a564e5944f311ff8","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1353bae1a564e5944f311ff8: start","job_status":"start","queue_duration_s":1.681916,"scheduling_latency_s":1.681928} gitlab # [ 143.272982] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.230Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1353bae1a564e5944f311ff8","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1353bae1a564e5944f311ff8: done: 0.00957 sec","job_status":"done","queue_duration_s":1.681916,"scheduling_latency_s":1.681928,"redis_calls":13,"redis_duration_s":0.001656,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001169,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00957,"completed_at":"2026-11-13T10:36:05.230Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.307676] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2aee6a695a08565565111dbe","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2aee6a695a08565565111dbe: start","job_status":"start","queue_duration_s":1.692292,"scheduling_latency_s":1.692304} gitlab # [ 143.348708] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2aee6a695a08565565111dbe","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2aee6a695a08565565111dbe: done: 0.009574 sec","job_status":"done","queue_duration_s":1.692292,"scheduling_latency_s":1.692304,"redis_calls":13,"redis_duration_s":0.001701,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009574,"completed_at":"2026-11-13T10:36:05.240Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.352235] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9870140786390f277c0a1a66","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9870140786390f277c0a1a66: start","job_status":"start","queue_duration_s":1.702664,"scheduling_latency_s":1.702677} gitlab # [ 143.363230] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9870140786390f277c0a1a66","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9870140786390f277c0a1a66: done: 0.009717 sec","job_status":"done","queue_duration_s":1.702664,"scheduling_latency_s":1.702677,"redis_calls":13,"redis_duration_s":0.00167,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001181,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006143,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009717,"completed_at":"2026-11-13T10:36:05.251Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.428110] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bc25b701bab4fd8a630a01a","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bc25b701bab4fd8a630a01a: start","job_status":"start","queue_duration_s":1.713214,"scheduling_latency_s":1.713228} gitlab # [ 143.428803] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bc25b701bab4fd8a630a01a","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bc25b701bab4fd8a630a01a: done: 0.009591 sec","job_status":"done","queue_duration_s":1.713214,"scheduling_latency_s":1.713228,"redis_calls":13,"redis_duration_s":0.0014830000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009591,"completed_at":"2026-11-13T10:36:05.261Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.442674] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c9d32259f5a80199dc877b0","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c9d32259f5a80199dc877b0: start","job_status":"start","queue_duration_s":1.723731,"scheduling_latency_s":1.723743} gitlab # [ 143.454098] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c9d32259f5a80199dc877b0","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c9d32259f5a80199dc877b0: done: 0.009598 sec","job_status":"done","queue_duration_s":1.723731,"scheduling_latency_s":1.723743,"redis_calls":13,"redis_duration_s":0.001484,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009598,"completed_at":"2026-11-13T10:36:05.272Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.492272] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ebaa513441d28febc39af950","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ebaa513441d28febc39af950: start","job_status":"start","queue_duration_s":1.734311,"scheduling_latency_s":1.734325} gitlab # [ 143.498522] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ebaa513441d28febc39af950","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ebaa513441d28febc39af950: done: 0.009599 sec","job_status":"done","queue_duration_s":1.734311,"scheduling_latency_s":1.734325,"redis_calls":13,"redis_duration_s":0.0015559999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009599,"completed_at":"2026-11-13T10:36:05.282Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.533145] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c903cfc7c4166858ceb74f8","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c903cfc7c4166858ceb74f8: start","job_status":"start","queue_duration_s":1.74489,"scheduling_latency_s":1.744902} gitlab # [ 143.544361] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.293Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c903cfc7c4166858ceb74f8","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c903cfc7c4166858ceb74f8: done: 0.009795 sec","job_status":"done","queue_duration_s":1.74489,"scheduling_latency_s":1.744902,"redis_calls":13,"redis_duration_s":0.001549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009795,"completed_at":"2026-11-13T10:36:05.293Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.596742] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.294Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2195ed472e1e4dedbcdf6383","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2195ed472e1e4dedbcdf6383: start","job_status":"start","queue_duration_s":1.75567,"scheduling_latency_s":1.755683} gitlab # [ 143.597628] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2195ed472e1e4dedbcdf6383","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2195ed472e1e4dedbcdf6383: done: 0.010059 sec","job_status":"done","queue_duration_s":1.75567,"scheduling_latency_s":1.755683,"redis_calls":13,"redis_duration_s":0.001639,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010059,"completed_at":"2026-11-13T10:36:05.304Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.640749] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7673847e0386c0d03021def7","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7673847e0386c0d03021def7: start","job_status":"start","queue_duration_s":1.76671,"scheduling_latency_s":1.766723} gitlab # [ 143.641896] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7673847e0386c0d03021def7","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7673847e0386c0d03021def7: done: 0.010185 sec","job_status":"done","queue_duration_s":1.76671,"scheduling_latency_s":1.766723,"redis_calls":13,"redis_duration_s":0.001765,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010185,"completed_at":"2026-11-13T10:36:05.315Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.701501] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c54dfd90e38d805cc783affd","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c54dfd90e38d805cc783affd: start","job_status":"start","queue_duration_s":1.777912,"scheduling_latency_s":1.777924} gitlab # [ 143.702476] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c54dfd90e38d805cc783affd","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c54dfd90e38d805cc783affd: done: 0.009794 sec","job_status":"done","queue_duration_s":1.777912,"scheduling_latency_s":1.777924,"redis_calls":13,"redis_duration_s":0.0015140000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009794,"completed_at":"2026-11-13T10:36:05.326Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.714269] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e53c63e6b2be37f404963e6","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e53c63e6b2be37f404963e6: start","job_status":"start","queue_duration_s":1.788533,"scheduling_latency_s":1.788546} gitlab # [ 143.725923] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.337Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e53c63e6b2be37f404963e6","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e53c63e6b2be37f404963e6: done: 0.0101 sec","job_status":"done","queue_duration_s":1.788533,"scheduling_latency_s":1.788546,"redis_calls":13,"redis_duration_s":0.001606,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006415,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0101,"completed_at":"2026-11-13T10:36:05.337Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.761239] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36d2e254923413b4b528bb20","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36d2e254923413b4b528bb20: start","job_status":"start","queue_duration_s":1.799518,"scheduling_latency_s":1.799531} gitlab # [ 143.772445] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36d2e254923413b4b528bb20","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36d2e254923413b4b528bb20: done: 0.009916 sec","job_status":"done","queue_duration_s":1.799518,"scheduling_latency_s":1.799531,"redis_calls":13,"redis_duration_s":0.0015119999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009916,"completed_at":"2026-11-13T10:36:05.348Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.829357] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8a6a97e84a30d350613b59f","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8a6a97e84a30d350613b59f: start","job_status":"start","queue_duration_s":1.810277,"scheduling_latency_s":1.810289} gitlab # [ 143.830140] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8a6a97e84a30d350613b59f","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8a6a97e84a30d350613b59f: done: 0.009986 sec","job_status":"done","queue_duration_s":1.810277,"scheduling_latency_s":1.810289,"redis_calls":13,"redis_duration_s":0.0015350000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009986,"completed_at":"2026-11-13T10:36:05.359Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.851679] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.360Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb6bbbb5eab6328167459024","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb6bbbb5eab6328167459024: start","job_status":"start","queue_duration_s":1.821114,"scheduling_latency_s":1.821127} gitlab # [ 143.862720] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb6bbbb5eab6328167459024","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb6bbbb5eab6328167459024: done: 0.009936 sec","job_status":"done","queue_duration_s":1.821114,"scheduling_latency_s":1.821127,"redis_calls":13,"redis_duration_s":0.0015170000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009936,"completed_at":"2026-11-13T10:36:05.370Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.908598] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.371Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4701fa8c25fb83122caab346","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4701fa8c25fb83122caab346: start","job_status":"start","queue_duration_s":1.831889,"scheduling_latency_s":1.831901} gitlab # [ 143.909487] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4701fa8c25fb83122caab346","created_at":"2026-11-13T10:36:03.372Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-325864725dc74222","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"750e947fb182de630a9d55af2bf63edc","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.539Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4701fa8c25fb83122caab346: done: 0.009645 sec","job_status":"done","queue_duration_s":1.831889,"scheduling_latency_s":1.831901,"redis_calls":13,"redis_duration_s":0.0017100000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001147,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009645,"completed_at":"2026-11-13T10:36:05.380Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.953507] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.381Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"52ae2de19c4e593a1fd01612","created_at":"2026-11-13T10:36:03.559Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb31a33dff5bdbaebcd824c7932be7f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.571Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-52ae2de19c4e593a1fd01612: start","job_status":"start","queue_duration_s":1.810201,"scheduling_latency_s":1.810214} gitlab # [ 143.954322] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"52ae2de19c4e593a1fd01612","created_at":"2026-11-13T10:36:03.559Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb31a33dff5bdbaebcd824c7932be7f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.571Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-52ae2de19c4e593a1fd01612: done: 0.022211 sec","job_status":"done","queue_duration_s":1.810201,"scheduling_latency_s":1.810214,"redis_calls":20,"redis_duration_s":0.002444,"redis_read_bytes":20,"redis_write_bytes":3248,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.000982,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1346,"redis_queues_calls":2,"redis_queues_duration_s":0.000228,"redis_queues_read_bytes":3,"redis_queues_write_bytes":818,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":734,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000361,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":350,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212596,"duration_s":0.022211,"completed_at":"2026-11-13T10:36:05.403Z","load_balancing_strategy":"primary","db_duration_s":0.001685,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.991145] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d1f9729bf7726b9e7aa3d806","created_at":"2026-11-13T10:36:03.629Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1540a683dcbf7361b5c0ba244b082978","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.650Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d1f9729bf7726b9e7aa3d806: start","job_status":"start","queue_duration_s":1.754428,"scheduling_latency_s":1.754442} gitlab # [ 144.003986] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.412Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 144.006910] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.417Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d1f9729bf7726b9e7aa3d806","created_at":"2026-11-13T10:36:03.629Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1540a683dcbf7361b5c0ba244b082978","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.650Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d1f9729bf7726b9e7aa3d806: done: 0.012992 sec","job_status":"done","queue_duration_s":1.754428,"scheduling_latency_s":1.754442,"redis_calls":10,"redis_duration_s":0.001282,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000236,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000207568,"duration_s":0.012992,"completed_at":"2026-11-13T10:36:05.417Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001348,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.066914] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.418Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5500e05cd1332cf001f95177","created_at":"2026-11-13T10:36:03.655Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"101a1819c9746d0ede13abc5894b4630","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.659Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5500e05cd1332cf001f95177: start","job_status":"start","queue_duration_s":1.759626,"scheduling_latency_s":1.759639} gitlab # [ 144.067600] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.435Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5500e05cd1332cf001f95177","created_at":"2026-11-13T10:36:03.655Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"101a1819c9746d0ede13abc5894b4630","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.659Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5500e05cd1332cf001f95177: done: 0.016886 sec","job_status":"done","queue_duration_s":1.759626,"scheduling_latency_s":1.759639,"redis_calls":12,"redis_duration_s":0.001601,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000235,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000575,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00033887,"exclusive_lock_hold_duration_s":0.003789588,"duration_s":0.016886,"completed_at":"2026-11-13T10:36:05.435Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003243,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.108726] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.436Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"c2aada5df5717e5d4ad0bdea","created_at":"2026-11-13T10:36:03.664Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"53d1a27cd114ca968e8a67cc5a260d49","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.679Z","job_size_bytes":3,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-c2aada5df5717e5d4ad0bdea: start","job_status":"start","queue_duration_s":1.757641,"scheduling_latency_s":1.757654} gitlab # [ 144.109865] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.448Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"c2aada5df5717e5d4ad0bdea","created_at":"2026-11-13T10:36:03.664Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"53d1a27cd114ca968e8a67cc5a260d49","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.679Z","job_size_bytes":3,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-c2aada5df5717e5d4ad0bdea: done: 0.012335 sec","job_status":"done","queue_duration_s":1.757641,"scheduling_latency_s":1.757654,"redis_calls":10,"redis_duration_s":0.001222,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000234,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000738,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00025,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000361777,"duration_s":0.012335,"completed_at":"2026-11-13T10:36:05.448Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001022,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.143258] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.450Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"70185673998ddc036783dcda","created_at":"2026-11-13T10:36:03.658Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-78f55f22ef994bb1","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8ddc8f39a75daae8bb98eea18ffe517b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.681Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-70185673998ddc036783dcda: start","job_status":"start","queue_duration_s":1.768877,"scheduling_latency_s":1.768889} gitlab # [ 144.183112] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.462Z","message":"No existing merge request to be cleaned up."} gitlab # [ 144.183703] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.471Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"70185673998ddc036783dcda","created_at":"2026-11-13T10:36:03.658Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-78f55f22ef994bb1","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8ddc8f39a75daae8bb98eea18ffe517b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.681Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-70185673998ddc036783dcda: done: 0.021256 sec","job_status":"done","queue_duration_s":1.768877,"scheduling_latency_s":1.768889,"redis_calls":13,"redis_duration_s":0.001607,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000241,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000625,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000741,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000195556,"duration_s":0.021256,"completed_at":"2026-11-13T10:36:05.471Z","load_balancing_strategy":"primary","db_duration_s":0.005521,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.224671] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.472Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9ab958370fdc3636e7cc01b0","created_at":"2026-11-13T10:36:03.658Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-78f55f22ef994bb1","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8ddc8f39a75daae8bb98eea18ffe517b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.681Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9ab958370fdc3636e7cc01b0: start","job_status":"start","queue_duration_s":1.791111,"scheduling_latency_s":1.791124} gitlab # [ 144.225650] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.478Z","message":"No existing merge request to be cleaned up."} gitlab # [ 144.226413] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.483Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9ab958370fdc3636e7cc01b0","created_at":"2026-11-13T10:36:03.658Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-78f55f22ef994bb1","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8ddc8f39a75daae8bb98eea18ffe517b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.681Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9ab958370fdc3636e7cc01b0: done: 0.010918 sec","job_status":"done","queue_duration_s":1.791111,"scheduling_latency_s":1.791124,"redis_calls":10,"redis_duration_s":0.001271,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00076,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010918,"completed_at":"2026-11-13T10:36:05.483Z","load_balancing_strategy":"primary","db_duration_s":0.000993,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.235548] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.484Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"84e23ab7e6cdfd63d7ad492d","created_at":"2026-11-13T10:36:03.658Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-78f55f22ef994bb1","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8ddc8f39a75daae8bb98eea18ffe517b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.681Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-84e23ab7e6cdfd63d7ad492d: start","job_status":"start","queue_duration_s":1.803026,"scheduling_latency_s":1.803039} gitlab # [ 144.247268] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.490Z","message":"No existing merge request to be cleaned up."} gitlab # [ 144.248732] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.495Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"84e23ab7e6cdfd63d7ad492d","created_at":"2026-11-13T10:36:03.658Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-78f55f22ef994bb1","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8ddc8f39a75daae8bb98eea18ffe517b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.681Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-84e23ab7e6cdfd63d7ad492d: done: 0.010925 sec","job_status":"done","queue_duration_s":1.803026,"scheduling_latency_s":1.803039,"redis_calls":10,"redis_duration_s":0.001291,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00074,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010925,"completed_at":"2026-11-13T10:36:05.495Z","load_balancing_strategy":"primary","db_duration_s":0.000956,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.303736] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.496Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b0d9c6a0e73d3bcea061b859","created_at":"2026-11-13T10:36:03.658Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-78f55f22ef994bb1","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8ddc8f39a75daae8bb98eea18ffe517b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.681Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b0d9c6a0e73d3bcea061b859: start","job_status":"start","queue_duration_s":1.815,"scheduling_latency_s":1.815013} gitlab # [ 144.304564] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.502Z","message":"No existing merge request to be cleaned up."} gitlab # [ 144.305372] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.507Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b0d9c6a0e73d3bcea061b859","created_at":"2026-11-13T10:36:03.658Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-78f55f22ef994bb1","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8ddc8f39a75daae8bb98eea18ffe517b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.681Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b0d9c6a0e73d3bcea061b859: done: 0.010918 sec","job_status":"done","queue_duration_s":1.815,"scheduling_latency_s":1.815013,"redis_calls":10,"redis_duration_s":0.001346,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000779,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007151,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010918,"completed_at":"2026-11-13T10:36:05.507Z","load_balancing_strategy":"primary","db_duration_s":0.000929,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.330570] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.507Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"6aa5722bcd0517f8b0e21dd1","created_at":"2026-11-13T10:36:03.701Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8938fb8a13167c7d1306e1756b075a4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.759Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-6aa5722bcd0517f8b0e21dd1: start","job_status":"start","queue_duration_s":1.747974,"scheduling_latency_s":1.747986} gitlab # [ 144.344154] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.520Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"6aa5722bcd0517f8b0e21dd1","created_at":"2026-11-13T10:36:03.701Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8938fb8a13167c7d1306e1756b075a4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.759Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-6aa5722bcd0517f8b0e21dd1: done: 0.012899 sec","job_status":"done","queue_duration_s":1.747974,"scheduling_latency_s":1.747986,"redis_calls":10,"redis_duration_s":0.001313,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000245,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000282,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000197791,"duration_s":0.012899,"completed_at":"2026-11-13T10:36:05.520Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000939,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.386220] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.521Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e2449bcb9f8611a0ec232e43","created_at":"2026-11-13T10:36:03.778Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1264be46c17c6ed0d77c5d230347f640","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-13T10:36:03.793Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e2449bcb9f8611a0ec232e43: start","job_status":"start","queue_duration_s":1.728041,"scheduling_latency_s":1.728053} gitlab # [ 144.393682] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.528Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 144.397124] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.533Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"e2449bcb9f8611a0ec232e43","created_at":"2026-11-13T10:36:03.778Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1264be46c17c6ed0d77c5d230347f640","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-13T10:36:03.793Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-e2449bcb9f8611a0ec232e43: done: 0.011445 sec","job_status":"done","queue_duration_s":1.728041,"scheduling_latency_s":1.728053,"redis_calls":10,"redis_duration_s":0.001179,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00023,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000698,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000189968,"duration_s":0.011445,"completed_at":"2026-11-13T10:36:05.533Z","load_balancing_strategy":"primary","db_duration_s":0.000306,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.470703] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.534Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"eaf18797edee0e3b35977b7a","created_at":"2026-11-13T10:36:03.848Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed997bdfe475d1396d9236a10771cafc","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.860Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-eaf18797edee0e3b35977b7a: start","job_status":"start","queue_duration_s":1.673109,"scheduling_latency_s":1.673121} gitlab # [ 144.471485] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.548Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"eaf18797edee0e3b35977b7a","created_at":"2026-11-13T10:36:03.848Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed997bdfe475d1396d9236a10771cafc","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.860Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-eaf18797edee0e3b35977b7a: done: 0.014411 sec","job_status":"done","queue_duration_s":1.673109,"scheduling_latency_s":1.673121,"redis_calls":10,"redis_duration_s":0.001191,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000231,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000698,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000262,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000191923,"duration_s":0.014411,"completed_at":"2026-11-13T10:36:05.548Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001412,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.511047] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.549Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"909e08d61c06ee2006267b7b","created_at":"2026-11-13T10:36:03.875Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e223ae88548eeca384b94144ddbb4181","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.884Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-909e08d61c06ee2006267b7b: start","job_status":"start","queue_duration_s":1.665148,"scheduling_latency_s":1.665161} gitlab # [ 144.511613] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.557Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 144.512102] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.562Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"909e08d61c06ee2006267b7b","created_at":"2026-11-13T10:36:03.875Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e223ae88548eeca384b94144ddbb4181","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.884Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-909e08d61c06ee2006267b7b: done: 0.013156 sec","job_status":"done","queue_duration_s":1.665148,"scheduling_latency_s":1.665161,"redis_calls":10,"redis_duration_s":0.001876,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000248,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001385,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000243,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000194438,"duration_s":0.013156,"completed_at":"2026-11-13T10:36:05.562Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001003,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.535473] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.563Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"21c316c3160acacb40254bc5","created_at":"2026-11-13T10:36:03.908Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78d023fe97c82a25728ab0d5bf716f64","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.914Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-21c316c3160acacb40254bc5: start","job_status":"start","queue_duration_s":1.648436,"scheduling_latency_s":1.648449} gitlab # [ 144.580177] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.570Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 144.580740] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.575Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"21c316c3160acacb40254bc5","created_at":"2026-11-13T10:36:03.908Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78d023fe97c82a25728ab0d5bf716f64","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.914Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-21c316c3160acacb40254bc5: done: 0.011844 sec","job_status":"done","queue_duration_s":1.648436,"scheduling_latency_s":1.648449,"redis_calls":10,"redis_duration_s":0.001178,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000236,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000695,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000247,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000193879,"duration_s":0.011844,"completed_at":"2026-11-13T10:36:05.575Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000756,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.587726] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.576Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Runners::StaleMachinesCleanupCronWorker","jid":"a134bf7674bb331b606d8760","created_at":"2026-11-13T10:36:03.924Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a5e3b625a2cffc71c7f4606fb1b023e","meta.root_caller_id":"Cronjob","meta.feature_category":"fleet_visibility","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b74a831dd4b14c3fc2b303407a0e209ea633ae1f03f57327e51b9e7be5871f1f","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.930Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Runners::StaleMachinesCleanupCronWorker JID-a134bf7674bb331b606d8760: start","job_status":"start","queue_duration_s":1.645774,"scheduling_latency_s":1.645786} gitlab # [ 144.628366] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.595Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Runners::StaleMachinesCleanupCronWorker","jid":"a134bf7674bb331b606d8760","created_at":"2026-11-13T10:36:03.924Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6a5e3b625a2cffc71c7f4606fb1b023e","meta.root_caller_id":"Cronjob","meta.feature_category":"fleet_visibility","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b74a831dd4b14c3fc2b303407a0e209ea633ae1f03f57327e51b9e7be5871f1f","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.930Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Runners::StaleMachinesCleanupCronWorker JID-a134bf7674bb331b606d8760: done: 0.019741 sec","job_status":"done","queue_duration_s":1.645774,"scheduling_latency_s":1.645786,"redis_calls":10,"redis_duration_s":0.001232,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000735,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000199466,"extra.ci_runners_stale_machines_cleanup_cron_worker.status":"success","extra.ci_runners_stale_machines_cleanup_cron_worker.total_deleted":0,"extra.ci_runners_stale_machines_cleanup_cron_worker.batch_counts":[0],"duration_s":0.019741,"completed_at":"2026-11-13T10:36:05.595Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005431,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.670024] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.596Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"8811a3e42281d818fdf2d05a","created_at":"2026-11-13T10:36:03.937Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"09cd4ffc22f562f7b5335087186e3481","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.944Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-8811a3e42281d818fdf2d05a: start","job_status":"start","queue_duration_s":1.65224,"scheduling_latency_s":1.652253} gitlab # [ 144.670829] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.645Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"8811a3e42281d818fdf2d05a","created_at":"2026-11-13T10:36:03.937Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"09cd4ffc22f562f7b5335087186e3481","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.944Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-8811a3e42281d818fdf2d05a: done: 0.048343 sec","job_status":"done","queue_duration_s":1.65224,"scheduling_latency_s":1.652253,"redis_calls":13,"redis_duration_s":0.001795,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000278,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000614,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006348851,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.048343,"completed_at":"2026-11-13T10:36:05.645Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029728,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 144.689548] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.646Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"2d8031b38a6ebdbc90903023","created_at":"2026-11-13T10:36:03.947Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7cd89c9e99b8084b293ba214fe188a8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.950Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-2d8031b38a6ebdbc90903023: start","job_status":"start","queue_duration_s":1.695465,"scheduling_latency_s":1.695479} gitlab # [ 144.731110] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.665Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"2d8031b38a6ebdbc90903023","created_at":"2026-11-13T10:36:03.947Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7cd89c9e99b8084b293ba214fe188a8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.950Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-2d8031b38a6ebdbc90903023: done: 0.01915 sec","job_status":"done","queue_duration_s":1.695465,"scheduling_latency_s":1.695479,"redis_calls":12,"redis_duration_s":0.004307,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000238,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003509,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00056,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00030339,"exclusive_lock_hold_duration_s":0.006790806999999999,"duration_s":0.01915,"completed_at":"2026-11-13T10:36:05.665Z","load_balancing_strategy":"primary","db_duration_s":0.001691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.737391] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.666Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"79954d436d8a7e0b1200d174","created_at":"2026-11-13T10:36:03.964Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"291b2c364d552b714045d028dcd47fea","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.971Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-79954d436d8a7e0b1200d174: start","job_status":"start","queue_duration_s":1.694658,"scheduling_latency_s":1.694671} gitlab # [ 144.786875] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.678Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"79954d436d8a7e0b1200d174","created_at":"2026-11-13T10:36:03.964Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"291b2c364d552b714045d028dcd47fea","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.971Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-79954d436d8a7e0b1200d174: done: 0.012304 sec","job_status":"done","queue_duration_s":1.694658,"scheduling_latency_s":1.694671,"redis_calls":12,"redis_duration_s":0.001802,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000243,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000831,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000728,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000194718,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.012304,"completed_at":"2026-11-13T10:36:05.678Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000239,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.789659] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.679Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"51348e880ab7ec5f430d3d52","created_at":"2026-11-13T10:36:03.978Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e54f328c20dbd98c185961990ea10ccc","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.983Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-51348e880ab7ec5f430d3d52: start","job_status":"start","queue_duration_s":1.696516,"scheduling_latency_s":1.696529} gitlab # [ 144.803096] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.693Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"51348e880ab7ec5f430d3d52","created_at":"2026-11-13T10:36:03.978Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-fec1371c88134b37","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e54f328c20dbd98c185961990ea10ccc","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:03.983Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-51348e880ab7ec5f430d3d52: done: 0.013697 sec","job_status":"done","queue_duration_s":1.696516,"scheduling_latency_s":1.696529,"redis_calls":12,"redis_duration_s":0.001524,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000235,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00072,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000569,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000328254,"exclusive_lock_hold_duration_s":0.00173346,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.013697,"completed_at":"2026-11-13T10:36:05.693Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000882,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.868152] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"e5d8d6e8c5392d9865dcbcec","created_at":"2026-11-13T10:36:05.393Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-789a4187ef28459f","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"cb31a33dff5bdbaebcd824c7932be7f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:05.398Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-e5d8d6e8c5392d9865dcbcec: start","job_status":"start","queue_duration_s":0.295716,"scheduling_latency_s":0.295728} gitlab # [ 144.869065] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:05.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"e5d8d6e8c5392d9865dcbcec","created_at":"2026-11-13T10:36:05.393Z","trace_propagation_headers":{"sentry-trace":"d295e97891474fa4ac26de64f4311e32-789a4187ef28459f","baggage":"sentry-trace_id=d295e97891474fa4ac26de64f4311e32,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"cb31a33dff5bdbaebcd824c7932be7f5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:05.398Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"maga","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-e5d8d6e8c5392d9865dcbcec: done: 0.019267 sec","job_status":"done","queue_duration_s":0.295716,"scheduling_latency_s":0.295728,"redis_calls":13,"redis_duration_s":0.001688,"redis_read_bytes":12,"redis_write_bytes":1665,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000235,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000589,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":519,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000864,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":801,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000199746,"duration_s":0.019267,"completed_at":"2026-11-13T10:36:05.713Z","load_balancing_strategy":"primary","db_duration_s":0.002296,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.077093] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2665997298 wd_nsec: 2665997210 gitlab # [ 155.848133] setup-gitlab-start[2190]: W, [2026-11-13T10:36:19.179118 #2190] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 176.908948] postgres[2386]: [2386] ERROR: relation "table_without_models" does not exist gitlab # [ 176.910104] postgres[2386]: [2386] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 180.261678] runuser[2189]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 180.264749] setup-gitlab-start[2169]: + systemctl restart gitlab-sidekiq.service gitlab # [ 180.285328] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 180.287963] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:43.616Z","message":"Shutting down"} gitlab # [ 180.291282] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:43.616Z","message":"Scheduler exiting..."} gitlab # [ 180.294577] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:43.617Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 180.298521] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:43.617Z","message":"Scheduler exiting..."} gitlab # [ 180.787554] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:44.118Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 182.791250] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:46.122Z","message":"Bye!"} gitlab # [ 182.814459] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-11-13T10:36:46.146Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1598,"worker_id":"sidekiq_0","memwd_rss_bytes":961961984,"retry":0} gitlab # [ 183.808676] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 183.809324] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 183.811118] systemd[1]: gitlab-sidekiq.service: Consumed 54.384s CPU time over 1min 52.981s wall clock time, 964.2M memory peak, 4.4M read from disk. gitlab # [ 183.820142] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 183.826844] setup-gitlab-start[2169]: + mkdir -p /var/lib/secrets gitlab # [ 183.827584] setup-gitlab-start[2169]: + echo nixbot-test-token-of-20-chars gitlab # [ 183.830780] setup-gitlab-start[2169]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 183.837093] setup-gitlab-start[2169]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars gitlab # [ 184.060087] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:47 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 195 gitlab # [ 184.568528] gitaly[906]: time="2026-11-13T10:36:47.899Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:36:57.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:47.898" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.590179] gitaly[906]: time="2026-11-13T10:36:47.899Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:36:57.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:47.898" grpc.time_ms=0.404 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.667276] gitaly[906]: time="2026-11-13T10:36:47.998Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:36:57.108" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:47.998" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.689322] gitaly[906]: time="2026-11-13T10:36:47.999Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:36:57.108" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:47.998" grpc.time_ms=0.417 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.772466] gitaly[906]: time="2026-11-13T10:36:48.104Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T diskcache=314fabc0-ea0e-4992-8142-850450dd62bd grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-13T10:36:58.009" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:48.050" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.789862] gitaly[906]: time="2026-11-13T10:36:48.121Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1574 command.oublock=32 command.real_time_ms=24 command.spawn_token_fork_ms=0 command.system_time_ms=16 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-13T10:36:58.009" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:48.050" grpc.time_ms=53.828 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.822097] gitaly[906]: time="2026-11-13T10:36:48.153Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:36:58.112" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:48.152" pid=906 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 184.822780] gitaly[906]: time="2026-11-13T10:36:48.154Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:36:58.112" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:48.152" grpc.time_ms=0.782 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.851976] gitaly[906]: time="2026-11-13T10:36:48.183Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:36:58.132" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:48.173" grpc.time_ms=10.484 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.914749] gitaly[906]: time="2026-11-13T10:36:48.246Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:36:58.205" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:48.246" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.915053] gitaly[906]: time="2026-11-13T10:36:48.246Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:36:58.205" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:48.246" grpc.time_ms=0.353 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.998158] gitaly[906]: time="2026-11-13T10:36:48.329Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T diskcache=2ec9d785-ce47-4d47-b8dd-16519c60d2a1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-13T10:36:58.246" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:48.287" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.998636] gitaly[906]: time="2026-11-13T10:36:48.330Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1582 command.oublock=32 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-13T10:36:58.246" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:48.287" grpc.time_ms=42.894 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 185.035553] gitaly[906]: time="2026-11-13T10:36:48.367Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:36:58.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:48.366" pid=906 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 185.036326] gitaly[906]: time="2026-11-13T10:36:48.367Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:36:58.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:48.366" grpc.time_ms=0.357 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 185.068675] gitaly[906]: time="2026-11-13T10:36:48.400Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:36:58.030" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:48.390" grpc.time_ms=9.903 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 185.112596] gitaly[906]: time="2026-11-13T10:36:48.444Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:36:58.083" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:48.444" grpc.time_ms=0.106 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 185.132658] gitaly[906]: time="2026-11-13T10:36:48.464Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-13T10:36:58.103" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:48.464" grpc.time_ms=0.152 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 185.660258] gitaly[906]: time="2026-11-13T10:36:48.991Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PDXMG9A5QCEBBQKT7S7T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:36:58.090" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:48.980" grpc.time_ms=10.201 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 185.881176] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:49 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 1798 gitlab # [ 185.887122] setup-gitlab-start[2440]: {"id":1,"message":null,"error":null} gitlab # [ 186.788475] sidekiq-cluster[2428]: {"severity":"INFO","time":"2026-11-13T10:36:50.103Z","message":"Starting cluster with 1 processes"} gitlab # [ 191.023307] gitaly[906]: time="2026-11-13T10:36:54.354Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PMK4R8KS82GHXYD5BBVG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:37:04.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:54.343" grpc.time_ms=10.231 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.093737] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:54 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 179 gitlab # [ 191.101484] setup-gitlab-start[2169]: + mkdir -p /tmp/test-flake gitlab # [ 191.103487] setup-gitlab-start[2169]: + cd /tmp/test-flake gitlab # [ 191.104488] setup-gitlab-start[2169]: + git init -b master gitlab # [ 191.133798] setup-gitlab-start[2514]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 191.136736] setup-gitlab-start[2169]: + git config user.name test gitlab # [ 191.140424] setup-gitlab-start[2169]: + git config user.email test@example.com gitlab # [ 191.144593] setup-gitlab-start[2169]: + cat gitlab # [ 191.149413] setup-gitlab-start[2169]: + git add flake.nix gitlab # [ 191.156076] setup-gitlab-start[2169]: + git commit -m 'initial commit' gitlab # [ 191.182077] setup-gitlab-start[2519]: [master (root-commit) 5feeafd] initial commit gitlab # [ 191.182736] setup-gitlab-start[2519]: 1 file changed, 10 insertions(+) gitlab # [ 191.184283] setup-gitlab-start[2519]: create mode 100644 flake.nix gitlab # [ 191.188146] setup-gitlab-start[2169]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 191.199352] setup-gitlab-start[2169]: + git push -u origin master gitlab # [ 191.521028] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:54 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 288 gitlab # [ 191.647179] gitaly[906]: time="2026-11-13T10:36:54.978Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PN6ABNA6SW0ARWA4JKYY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:36:54.968" grpc.time_ms=10.307 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 191.665258] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:54 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 138 gitlab # [ 191.866774] gitaly[906]: time="2026-11-13T10:36:55.198Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:37:05.147" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:55.188" grpc.time_ms=10.113 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.926465] gitaly[906]: time="2026-11-13T10:36:55.258Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-13T10:37:25.050" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-13T10:36:55.251" grpc.time_ms=6.95 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.969963] gitaly[906]: time="2026-11-13T10:36:55.301Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT duration_ms=164ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 192.015276] gitaly[906]: time="2026-11-13T10:36:55.346Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT duration_ms=44ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 192.021983] gitaly[906]: time="2026-11-13T10:36:55.353Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:36:55.136" grpc.time_ms=210.9 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 192.081904] gitaly[906]: time="2026-11-13T10:36:55.413Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:36:55.412" grpc.time_ms=0.822 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 192.144768] gitaly[906]: time="2026-11-13T10:36:55.476Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:36:55.475" grpc.time_ms=0.57 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 192.193566] gitaly[906]: time="2026-11-13T10:36:55.524Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:36:55.523" grpc.time_ms=0.545 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 192.280071] gitaly[906]: time="2026-11-13T10:36:55.611Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:37:05.241" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:36:55.601" grpc.time_ms=9.843 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.327569] gitaly[906]: time="2026-11-13T10:36:55.659Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1566 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-13T10:37:05.281" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:36:55.641" grpc.time_ms=17.275 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.347813] gitaly[906]: time="2026-11-13T10:36:55.679Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT duration_ms=124ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 192.350921] gitaly[906]: time="2026-11-13T10:36:55.682Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:36:55.554" grpc.time_ms=127.943 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 192.373538] gitaly[906]: time="2026-11-13T10:36:55.705Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT diskcache=de0f046f-dc76-49ea-b220-aea90182b2b7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:36:55.066" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 192.374858] gitlab-workhorse[827]: time="2026-11-13T10:36:55Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 192.375675] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:55 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 699 gitlab # [ 192.386114] gitaly[906]: time="2026-11-13T10:36:55.705Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=236 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13509 command.oublock=32 command.real_time_ms=629 command.spawn_token_fork_ms=0 command.system_time_ms=152 command.user_time_ms=84 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:36:55.066" grpc.time_ms=638.919 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 192.398361] setup-gitlab-start[2523]: To http://gitlab/root/test-flake.git gitlab # [ 192.401674] setup-gitlab-start[2523]: * [new branch] master -> master gitlab # [ 192.403185] setup-gitlab-start[2523]: branch 'master' set up to track 'origin/master'. gitlab # [ 192.405993] systemd[1]: Finished setup-gitlab.service. gitlab # [ 192.411530] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 193.46 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 193.511402] nixbot[2602]: {"ts": "2026-11-13T10:36:56.842Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 193.537785] nixbot[2602]: {"ts": "2026-11-13T10:36:56.869Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 193.568916] nixbot[2602]: {"ts": "2026-11-13T10:36:56.900Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 193.578699] nixbot[2602]: {"ts": "2026-11-13T10:36:56.910Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 193.589251] nixbot[2602]: {"ts": "2026-11-13T10:36:56.920Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 193.598273] nixbot[2602]: {"ts": "2026-11-13T10:36:56.929Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 193.607979] nixbot[2602]: {"ts": "2026-11-13T10:36:56.939Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 193.611708] nixbot[2602]: {"ts": "2026-11-13T10:36:56.943Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 193.620547] nixbot[2602]: {"ts": "2026-11-13T10:36:56.952Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 193.628302] nixbot[2602]: {"ts": "2026-11-13T10:36:56.959Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 193.640408] nixbot[2602]: {"ts": "2026-11-13T10:36:56.972Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 193.649910] nixbot[2602]: {"ts": "2026-11-13T10:36:56.981Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 193.658741] nixbot[2602]: {"ts": "2026-11-13T10:36:56.990Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 193.666665] nixbot[2602]: {"ts": "2026-11-13T10:36:56.998Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 193.672289] nixbot[2602]: {"ts": "2026-11-13T10:36:57.003Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 193.680435] nixbot[2602]: {"ts": "2026-11-13T10:36:57.012Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 193.683616] nixbot[2602]: {"ts": "2026-11-13T10:36:57.015Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 193.691670] nixbot[2602]: {"ts": "2026-11-13T10:36:57.023Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 193.701756] nixbot[2602]: {"ts": "2026-11-13T10:36:57.033Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 193.704717] nixbot[2602]: {"ts": "2026-11-13T10:36:57.036Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 193.707513] nixbot[2602]: {"ts": "2026-11-13T10:36:57.039Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 193.714874] nixbot[2602]: {"ts": "2026-11-13T10:36:57.046Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 193.721666] nixbot[2602]: {"ts": "2026-11-13T10:36:57.053Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 193.728692] nixbot[2602]: {"ts": "2026-11-13T10:36:57.060Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 193.734711] nixbot[2602]: {"ts": "2026-11-13T10:36:57.066Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 193.742428] nixbot[2602]: {"ts": "2026-11-13T10:36:57.073Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 193.745819] nixbot[2602]: {"ts": "2026-11-13T10:36:57.077Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 193.752638] nixbot[2602]: {"ts": "2026-11-13T10:36:57.084Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 193.758992] nixbot[2602]: {"ts": "2026-11-13T10:36:57.090Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 193.779299] nixbot[2602]: {"ts": "2026-11-13T10:36:57.110Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 193.789834] nixbot[2602]: {"ts": "2026-11-13T10:36:57.121Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 193.797309] nixbot[2602]: {"ts": "2026-11-13T10:36:57.128Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 193.804145] nixbot[2602]: {"ts": "2026-11-13T10:36:57.135Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} gitlab # [ 193.814703] nixbot[2602]: {"ts": "2026-11-13T10:36:57.146Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"} gitlab # [ 193.818181] nixbot[2602]: {"ts": "2026-11-13T10:36:57.149Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"} gitlab # [ 193.826664] nixbot[2602]: {"ts": "2026-11-13T10:36:57.158Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"} gitlab # [ 193.834533] nixbot[2602]: {"ts": "2026-11-13T10:36:57.166Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"} gitlab # [ 193.843687] nixbot[2602]: {"ts": "2026-11-13T10:36:57.174Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "effect_when_changed"} gitlab # [ 193.851433] nixbot[2602]: {"ts": "2026-11-13T10:36:57.183Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 38, "migration": "commit_statuses"} gitlab # [ 194.309797] nixbot[2602]: {"ts": "2026-11-13T10:36:57.641Z", "level": "info", "logger": "uvicorn.error", "msg": "Started server process [2602]", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"} gitlab # [ 194.310465] nixbot[2602]: {"ts": "2026-11-13T10:36:57.641Z", "level": "info", "logger": "uvicorn.error", "msg": "Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)"} gitlab # [ 194.347499] nixbot[2602]: {"ts": "2026-11-13T10:36:57.679Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47762 - \"GET /health HTTP/1.1\" 200"} gitlab # [ 194.354263] systemd[1]: Started nixbot CI service. gitlab # [ 194.354682] systemd[1]: Reached target Multi-User System. gitlab # [ 194.357417] systemd[1]: Startup finished in 1.191s (kernel) + 3.532s (initrd) + 3min 9.633s (userspace) = 3min 14.357s. gitlab: (finished: waiting for unit nixbot.service, in 2.09 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 194.930772] nixbot[2602]: {"ts": "2026-11-13T10:36:58.261Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:43320 - \"GET /health HTTP/1.1\" 200"} gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 195.013075] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 42 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.08 seconds) gitlab # [ 195.160805] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:58 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 814 gitlab # [ 195.177704] nixbot[2602]: {"ts": "2026-11-13T10:36:58.509Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 195.238333] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 45 gitlab # [ 195.241735] nixbot[2602]: {"ts": "2026-11-13T10:36:58.573Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 195.333332] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 85 gitlab # [ 195.470062] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:58 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 129 gitlab # [ 195.549815] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 62 gitlab # [ 195.602570] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:58 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 42 gitlab # [ 195.658256] gitaly[906]: time="2026-11-13T10:36:58.989Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PS62Q57W118TWC1F7DRJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-11-13T10:37:28.776" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:36:58.977" grpc.time_ms=12.069 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 195.668762] gitaly[906]: time="2026-11-13T10:36:59.000Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PS62Q57W118TWC1F7DRJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:37:28.798" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:36:58.999" grpc.time_ms=0.617 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 195.719759] gitaly[906]: time="2026-11-13T10:36:59.051Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PS62Q57W118TWC1F7DRJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-11-13T10:37:09.002" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:36:59.042" grpc.time_ms=8.448 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 195.738854] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 123 gitlab # [ 195.797905] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 53 gitlab # [ 195.809269] nixbot[2602]: {"ts": "2026-11-13T10:36:59.140Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 195.942566] gitaly[906]: time="2026-11-13T10:36:59.274Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PSE6TNP0W393BG1AZANZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:36:59.264" grpc.time_ms=9.56 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 195.974245] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:59 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 99 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 196.105171] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:59 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 47 gitlab # [ 196.109580] gitaly[906]: time="2026-11-13T10:36:59.441Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PSHBHF8MQQKX28TFAN1E grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:36:59.427" pid=906 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 196.133551] gitaly[906]: time="2026-11-13T10:36:59.465Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PSHBHF8MQQKX28TFAN1E grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:36:59.427" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.14 seconds) (finished: subtest: project discovered and webhook registered, in 1.22 seconds) subtest: push delivers the webhook gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' gitlab # [ 196.156604] gitaly[906]: time="2026-11-13T10:36:59.488Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PSHBHF8MQQKX28TFAN1E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:36:59.427" grpc.time_ms=60.999 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 196.157223] gitlab-workhorse[827]: time="2026-11-13T10:36:59Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 196.157984] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:59 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 181 gitlab # [ 196.321961] gitaly[906]: time="2026-11-13T10:36:59.651Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PSQJAQ6ZR82GV9ASRRC8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:36:59.632" pid=906 remote_ip="2001:db8:1::1" request_sha=83b85c5b32bf1b3ad2f8649540afd4569c0ff363 response_bytes=224 span.kind=server system=grpc gitlab # [ 196.322848] gitaly[906]: time="2026-11-13T10:36:59.652Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1200 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PSQJAQ6ZR82GV9ASRRC8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:36:59.632" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 196.341163] gitlab-workhorse[827]: time="2026-11-13T10:36:59Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=224 gitlab # [ 196.341804] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:59 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 224 "" "git/2.55.0" 145 gitlab # [ 196.342944] gitaly[906]: time="2026-11-13T10:36:59.652Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1200 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PSQJAQ6ZR82GV9ASRRC8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:36:59.632" grpc.time_ms=19.787 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 196.454027] gitaly[906]: time="2026-11-13T10:36:59.785Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PSYY5RZB27CJDCZC3GC0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:36:59.775" grpc.time_ms=9.612 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 196.479785] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:59 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 69 gitlab # [ 196.529652] gitaly[906]: time="2026-11-13T10:36:59.861Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PT17BS3EARNVAXP68WBB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:36:59.851" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 196.551244] gitaly[906]: time="2026-11-13T10:36:59.861Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PT17BS3EARNVAXP68WBB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:36:59.851" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 196.572734] gitaly[906]: time="2026-11-13T10:36:59.904Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PT17BS3EARNVAXP68WBB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:36:59.851" grpc.time_ms=52.541 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 196.585846] gitlab-workhorse[827]: time="2026-11-13T10:36:59Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 196.586468] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:36:59 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 101 gitlab # [ 196.659359] gitaly[906]: time="2026-11-13T10:36:59.990Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PT5BZYB1RF9C3YHZ946Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:36:59.980" grpc.time_ms=9.84 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 196.682930] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:00 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 66 gitlab # [ 196.733090] gitaly[906]: time="2026-11-13T10:37:00.064Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PT7HER1ANJ2F8NE44G5M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:00.050" pid=906 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 196.734213] gitaly[906]: time="2026-11-13T10:37:00.064Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1171 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PT7HER1ANJ2F8NE44G5M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:00.050" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 196.736118] gitlab-workhorse[827]: time="2026-11-13T10:37:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 196.737161] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 47 gitlab # [ 196.738467] gitaly[906]: time="2026-11-13T10:37:00.064Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1171 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PT7HER1ANJ2F8NE44G5M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:00.050" grpc.time_ms=13.829 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 196.854548] gitaly[906]: time="2026-11-13T10:37:00.186Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:37:30.084" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:00.185" grpc.time_ms=0.702 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.890353] gitaly[906]: time="2026-11-13T10:37:00.222Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-13T10:37:30.114" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:00.214" grpc.time_ms=7.152 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.914411] gitaly[906]: time="2026-11-13T10:37:00.246Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:10.199" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:00.239" grpc.time_ms=6.385 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.925556] gitaly[906]: time="2026-11-13T10:37:00.257Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:10.211" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:00.251" grpc.time_ms=5.545 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.988741] gitaly[906]: time="2026-11-13T10:37:00.319Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-13T10:37:10.279" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:00.319" grpc.time_ms=0.314 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.007411] gitaly[906]: time="2026-11-13T10:37:00.339Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-13T10:37:10.293" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:00.333" grpc.time_ms=5.418 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.062308] gitaly[906]: time="2026-11-13T10:37:00.394Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:00.393" grpc.time_ms=0.28 pid=906 span.kind=server system=grpc gitlab # [ 197.088961] gitaly[906]: time="2026-11-13T10:37:00.420Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:00.420" grpc.time_ms=0.222 pid=906 span.kind=server system=grpc gitlab # [ 197.106469] gitaly[906]: time="2026-11-13T10:37:00.438Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M diskcache=3f9e0982-eef3-4469-a01c-16c51a17a937 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:37:10.019" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:00.360" pid=906 ref=refs/keep-around/5feeafd1ab71e41915719113facb270437fd8a7e remote_ip="2001:db8:1::1" revision=5feeafd1ab71e41915719113facb270437fd8a7e span.kind=server system=grpc username=root gitlab # [ 197.107128] gitaly[906]: time="2026-11-13T10:37:00.438Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=55 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5389 command.oublock=8 command.real_time_ms=70 command.spawn_token_fork_ms=0 command.system_time_ms=34 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:37:10.019" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:00.360" grpc.time_ms=78.135 pid=906 ref=refs/keep-around/5feeafd1ab71e41915719113facb270437fd8a7e remote_ip="2001:db8:1::1" revision=5feeafd1ab71e41915719113facb270437fd8a7e span.kind=server system=grpc username=root gitlab # [ 197.130417] gitaly[906]: time="2026-11-13T10:37:00.462Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:37:30.361" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:00.461" grpc.time_ms=0.357 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.270965] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:00 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5feeafd1ab71e41915719113facb270437fd8a7e HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 455 gitlab # [ 197.285903] nixbot[2602]: {"ts": "2026-11-13T10:37:00.616Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-5feeafd1ab71-0"} gitlab # [ 197.497764] nixbot[2602]: {"ts": "2026-11-13T10:37:00.829Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-5feeafd1ab71-0", "argv0": "bwrap"} gitlab # [ 197.590914] systemd[1]: Started Nix Daemon. gitlab # [ 197.658459] nix-daemon[2765]: accepted connection from pid 2760, user nixbot gitlab # [ 202.163510] nix-daemon[2765]: accepted connection from pid 2807, user nixbot gitlab # [ 202.176546] nix-daemon[2765]: accepted connection from pid 2760, user nixbot gitlab # [ 206.450672] nix-daemon[2765]: accepted connection from pid 2760, user nixbot gitlab # [ 206.551126] nix-daemon[2765]: accepted connection from pid 2849, user nixbot gitlab # [ 206.577236] gitaly[906]: time="2026-11-13T10:37:09.908Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7Q3VEF7NAXP5GA2ZGYW7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:37:39.807" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:09.907" grpc.time_ms=0.504 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.664800] gitaly[906]: time="2026-11-13T10:37:09.996Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7Q3VEF7NAXP5GA2ZGYW7Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:19.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:09.985" grpc.time_ms=10.652 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.721719] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:10 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5feeafd1ab71e41915719113facb270437fd8a7e HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 182 gitlab # [ 206.784525] gitaly[906]: time="2026-11-13T10:37:10.116Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7Q41ARQ13TZP9H9Z5J2DE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:37:40.015" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:10.115" grpc.time_ms=0.458 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 206.924636] gitaly[906]: time="2026-11-13T10:37:10.256Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7Q41ARQ13TZP9H9Z5J2DE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:20.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:10.246" grpc.time_ms=9.883 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.008303] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:10 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5feeafd1ab71e41915719113facb270437fd8a7e HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 277 gitlab # [ 210.779163] sidekiq-cluster[2485]: 2026-11-13T10:37:14.110Z pid=2485 tid=xst INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2485, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>432975872} gitlab # [ 210.920265] nix-daemon[2765]: accepted connection from pid 2934, user nixbot gitlab # [ 210.943411] sidekiq-cluster[2485]: W, [2026-11-13T10:37:14.275155 #2485] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 210.981029] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:14.311Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 210.990120] gitaly[906]: time="2026-11-13T10:37:14.321Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7Q8529SQQ8JEVTP64QGFE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:37:44.220" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:14.320" grpc.time_ms=0.621 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.027175] gitaly[906]: time="2026-11-13T10:37:14.358Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7Q8529SQQ8JEVTP64QGFE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:24.108" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:14.349" grpc.time_ms=9.673 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 211.061279] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:14 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5feeafd1ab71e41915719113facb270437fd8a7e HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 116 gitlab # [ 211.237664] nixbot[2602]: {"ts": "2026-11-13T10:37:14.568Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 1, "sha": "5feeafd1ab71e41915719113facb270437fd8a7e", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []} gitlab # [ 211.309906] gitaly[906]: time="2026-11-13T10:37:14.641Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7Q8F6MC3WA1K9BY3DM4FY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:14.630" grpc.time_ms=10.996 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.328099] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:14 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 60 gitlab # [ 211.377637] gitaly[906]: time="2026-11-13T10:37:14.709Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7Q8H6ABK7Q05Z4EN0DR56 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:14.698" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 211.398943] gitaly[906]: time="2026-11-13T10:37:14.709Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7Q8H6ABK7Q05Z4EN0DR56 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:14.698" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.416533] gitlab-workhorse[827]: time="2026-11-13T10:37:14Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 211.416774] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:14 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 47 gitlab # [ 211.417448] gitaly[906]: time="2026-11-13T10:37:14.709Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7Q8H6ABK7Q05Z4EN0DR56 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:14.698" grpc.time_ms=11.317 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.499889] gitaly[906]: time="2026-11-13T10:37:14.831Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7Q8N3RKJZF7PZHYSZX19G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:14.824" grpc.time_ms=7.007 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.500193] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:14 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 44 gitlab # [ 211.560658] gitaly[906]: time="2026-11-13T10:37:14.892Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7Q8Q828D9CK4Y9SMQBPE1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:14.886" pid=906 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 211.560866] gitaly[906]: time="2026-11-13T10:37:14.892Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7Q8Q828D9CK4Y9SMQBPE1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:14.886" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.561979] gitaly[906]: time="2026-11-13T10:37:14.892Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7Q8Q828D9CK4Y9SMQBPE1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:14.886" grpc.time_ms=5.91 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 211.563130] gitlab-workhorse[827]: time="2026-11-13T10:37:14Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 211.563802] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:14 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 36 gitlab # [ 212.619079] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:15.949Z","message":"GitLab reliable fetch activated!"} gitlab # [ 212.620214] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:15.950Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 231.666529] postgres[3147]: [3147] ERROR: relation "table_without_models" does not exist gitlab # [ 231.666872] postgres[3147]: [3147] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 234.091909] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:37.423Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 234.093619] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:37.425Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 234.094717] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:37.425Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 234.095799] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:37.425Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 234.097209] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:37.428Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 38.24 seconds) gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 234.485808] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:37 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 39 gitlab # [ 234.582461] gitaly[906]: time="2026-11-13T10:37:37.913Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ4T3HNM9B04TECPJD3G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:37.901" grpc.time_ms=11.645 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 234.611968] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:37 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 95 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 273.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 234.800963] gitaly[906]: time="2026-11-13T10:37:38.132Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-13T10:38:08.022" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-13T10:37:38.122" grpc.time_ms=9.772 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 234.841897] gitaly[906]: time="2026-11-13T10:37:38.173Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M7A7QZ968R6WB2HBDDBP3611 duration_ms=93ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 234.871984] gitaly[906]: time="2026-11-13T10:37:38.203Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M7A7QZ968R6WB2HBDDBP3611 duration_ms=29ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 234.872993] gitaly[906]: time="2026-11-13T10:37:38.204Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:38.079" grpc.time_ms=124.814 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 234.942893] gitaly[906]: time="2026-11-13T10:37:38.274Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:38.273" grpc.time_ms=0.775 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 234.990622] gitaly[906]: time="2026-11-13T10:37:38.322Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:38.321" grpc.time_ms=0.276 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 235.028335] gitaly[906]: time="2026-11-13T10:37:38.360Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:38.359" grpc.time_ms=0.315 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 235.118573] gitaly[906]: time="2026-11-13T10:37:38.450Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M7A7QZ968R6WB2HBDDBP3611 duration_ms=56ns method=POST pid=906 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 235.119433] gitaly[906]: time="2026-11-13T10:37:38.451Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:38.393" grpc.time_ms=57.254 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 235.153724] gitaly[906]: time="2026-11-13T10:37:38.485Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 diskcache=722dacba-0a06-4ac3-b826-966118e0adfa grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:38.015" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 235.153912] gitaly[906]: time="2026-11-13T10:37:38.485Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=228 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=876544 command.minflt=13599 command.oublock=32 command.real_time_ms=462 command.spawn_token_fork_ms=0 command.system_time_ms=154 command.user_time_ms=74 component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:38.015" grpc.time_ms=470.04 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 235.155422] gitlab-workhorse[827]: time="2026-11-13T10:37:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 235.156444] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:38 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 527 gitlab # To http://gitlab/root/test-flake.git gitlab # 5feeafd..bd2d78b master -> master gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.81 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340' gitlab # [ 235.258176] nixbot[2602]: {"ts": "2026-11-13T10:37:38.589Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:37900 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 HTTP/1.1\" 200"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340', in 0.06 seconds) gitlab # [ 235.371592] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:38.702Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"ccd8f6f7b70eae66a418f9fc","created_at":"2026-11-13T10:36:48.194Z","trace_propagation_headers":{"sentry-trace":"4f877e63aa0a4483bcafb480013ef734-bb3ebe69fd984bc1","baggage":"sentry-trace_id=4f877e63aa0a4483bcafb480013ef734,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PDXMG9A5QCEBBQKT7S7T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:48.230Z","job_size_bytes":2,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-ccd8f6f7b70eae66a418f9fc: start","job_status":"start","queue_duration_s":50.472256,"scheduling_latency_s":50.472295} gitlab # [ 235.492671] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:38.824Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"ccd8f6f7b70eae66a418f9fc","created_at":"2026-11-13T10:36:48.194Z","trace_propagation_headers":{"sentry-trace":"4f877e63aa0a4483bcafb480013ef734-bb3ebe69fd984bc1","baggage":"sentry-trace_id=4f877e63aa0a4483bcafb480013ef734,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PDXMG9A5QCEBBQKT7S7T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:48.230Z","job_size_bytes":2,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-ccd8f6f7b70eae66a418f9fc: done: 0.121522 sec","job_status":"done","queue_duration_s":50.472256,"scheduling_latency_s":50.472295,"redis_calls":25,"redis_duration_s":0.003408,"redis_read_bytes":912,"redis_write_bytes":2805,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.001919,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000655,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":11,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.063624567,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.121522,"completed_at":"2026-11-13T10:37:38.824Z","load_balancing_strategy":"primary","db_duration_s":0.040529,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 235.493747] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:38.825Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"3f67bc80ad2a5c45a3e390bb","created_at":"2026-11-13T10:36:48.238Z","trace_propagation_headers":{"sentry-trace":"4f877e63aa0a4483bcafb480013ef734-bb3ebe69fd984bc1","baggage":"sentry-trace_id=4f877e63aa0a4483bcafb480013ef734,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PDXMG9A5QCEBBQKT7S7T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:48.242Z","job_size_bytes":2,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-3f67bc80ad2a5c45a3e390bb: start","job_status":"start","queue_duration_s":50.583209,"scheduling_latency_s":50.583224} gitlab # [ 235.569229] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:38.893Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"3f67bc80ad2a5c45a3e390bb","created_at":"2026-11-13T10:36:48.238Z","trace_propagation_headers":{"sentry-trace":"4f877e63aa0a4483bcafb480013ef734-bb3ebe69fd984bc1","baggage":"sentry-trace_id=4f877e63aa0a4483bcafb480013ef734,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PDXMG9A5QCEBBQKT7S7T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:48.242Z","job_size_bytes":2,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-3f67bc80ad2a5c45a3e390bb: done: 0.068411 sec","job_status":"done","queue_duration_s":50.583209,"scheduling_latency_s":50.583224,"redis_calls":14,"redis_duration_s":0.002774,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000903,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00104,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000831,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.053338343,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.068411,"completed_at":"2026-11-13T10:37:38.893Z","load_balancing_strategy":"primary","db_duration_s":0.018228,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 235.582486] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:38.894Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"c0cdf95ca52b72a34aa1c23a","created_at":"2026-11-13T10:36:48.850Z","trace_propagation_headers":{"sentry-trace":"4f877e63aa0a4483bcafb480013ef734-bb3ebe69fd984bc1","baggage":"sentry-trace_id=4f877e63aa0a4483bcafb480013ef734,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PDXMG9A5QCEBBQKT7S7T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:48.855Z","job_size_bytes":5,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-c0cdf95ca52b72a34aa1c23a: start","job_status":"start","queue_duration_s":50.039371,"scheduling_latency_s":50.039384} gitlab # [ 235.672594] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:39.003Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"c0cdf95ca52b72a34aa1c23a","created_at":"2026-11-13T10:36:48.850Z","trace_propagation_headers":{"sentry-trace":"4f877e63aa0a4483bcafb480013ef734-bb3ebe69fd984bc1","baggage":"sentry-trace_id=4f877e63aa0a4483bcafb480013ef734,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PDXMG9A5QCEBBQKT7S7T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:48.855Z","job_size_bytes":5,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-c0cdf95ca52b72a34aa1c23a: done: 0.109077 sec","job_status":"done","queue_duration_s":50.039371,"scheduling_latency_s":50.039384,"redis_calls":12,"redis_duration_s":0.002146,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000489,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001338,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000319,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000307302,"duration_s":0.109077,"completed_at":"2026-11-13T10:37:39.003Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023532,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 235.673794] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:39.005Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"e4da7bad1b4f86a6f597eaf3","created_at":"2026-11-13T10:36:48.891Z","trace_propagation_headers":{"sentry-trace":"4f877e63aa0a4483bcafb480013ef734-bb3ebe69fd984bc1","baggage":"sentry-trace_id=4f877e63aa0a4483bcafb480013ef734,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PDXMG9A5QCEBBQKT7S7T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:48.896Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-e4da7bad1b4f86a6f597eaf3: start","job_status":"start","queue_duration_s":50.108845,"scheduling_latency_s":50.108859} gitlab # [ 235.828214] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:39.159Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"e4da7bad1b4f86a6f597eaf3","created_at":"2026-11-13T10:36:48.891Z","trace_propagation_headers":{"sentry-trace":"4f877e63aa0a4483bcafb480013ef734-bb3ebe69fd984bc1","baggage":"sentry-trace_id=4f877e63aa0a4483bcafb480013ef734,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PDXMG9A5QCEBBQKT7S7T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:48.896Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-e4da7bad1b4f86a6f597eaf3: done: 0.153941 sec","job_status":"done","queue_duration_s":50.108845,"scheduling_latency_s":50.108859,"redis_calls":11,"redis_duration_s":0.0034809999999999997,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000789,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001952,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00074,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.123,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.123,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000660699,"duration_s":0.153941,"completed_at":"2026-11-13T10:37:39.159Z","load_balancing_strategy":"primary","db_duration_s":0.06987,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.830512] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:39.161Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"182b52aef00ff84a868206b6","created_at":"2026-11-13T10:36:48.900Z","trace_propagation_headers":{"sentry-trace":"4f877e63aa0a4483bcafb480013ef734-bb3ebe69fd984bc1","baggage":"sentry-trace_id=4f877e63aa0a4483bcafb480013ef734,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PDXMG9A5QCEBBQKT7S7T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:48.907Z","job_size_bytes":89,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-182b52aef00ff84a868206b6: start","job_status":"start","queue_duration_s":50.254222,"scheduling_latency_s":50.25426} gitlab # [ 235.912673] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:39.201Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"182b52aef00ff84a868206b6","created_at":"2026-11-13T10:36:48.900Z","trace_propagation_headers":{"sentry-trace":"4f877e63aa0a4483bcafb480013ef734-bb3ebe69fd984bc1","baggage":"sentry-trace_id=4f877e63aa0a4483bcafb480013ef734,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PDXMG9A5QCEBBQKT7S7T","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-13T10:36:48.907Z","job_size_bytes":89,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-182b52aef00ff84a868206b6: done: 0.0399 sec","job_status":"done","queue_duration_s":50.254222,"scheduling_latency_s":50.25426,"redis_calls":11,"redis_duration_s":0.002215,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000595,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001225,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000395,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000283556,"duration_s":0.0399,"completed_at":"2026-11-13T10:37:39.201Z","load_balancing_strategy":"primary","db_duration_s":0.004804,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.916504] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:39.202Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 5feeafd1ab71e41915719113facb270437fd8a7e refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"2420aca0cac16d9f01c96596","created_at":"2026-11-13T10:36:55.589Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-d105d3f8f92a48a8","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:55.594Z","job_size_bytes":185,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-2420aca0cac16d9f01c96596: start","job_status":"start","queue_duration_s":43.608099,"scheduling_latency_s":43.608112} gitlab # [ 235.931119] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:39.213Z","message":"changes: \"0000000000000000000000000000000000000000 5feeafd1ab71e41915719113facb270437fd8a7e refs/heads/master\\n\""} gitlab # [ 235.933505] gitaly[906]: time="2026-11-13T10:37:39.243Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:49.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:39.237" grpc.time_ms=5.504 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 235.979975] gitaly[906]: time="2026-11-13T10:37:39.257Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:37:49.018" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:39.248" grpc.time_ms=8.026 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 235.980633] gitaly[906]: time="2026-11-13T10:37:39.295Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-13T10:37:49.058" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:39.289" grpc.time_ms=6.068 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 235.988144] gitaly[906]: time="2026-11-13T10:37:39.319Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:39.318" grpc.time_ms=1.138 pid=906 span.kind=server system=grpc gitlab # [ 236.007247] gitaly[906]: time="2026-11-13T10:37:39.338Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:39.338" grpc.time_ms=0.455 pid=906 span.kind=server system=grpc gitlab # [ 236.017782] gitaly[906]: time="2026-11-13T10:37:39.349Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT diskcache=3e05b97c-bce8-4408-8b8d-71a67937ed83 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:37:49.070" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:39.301" pid=906 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 236.018653] gitaly[906]: time="2026-11-13T10:37:39.349Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=44 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4975 command.oublock=8 command.real_time_ms=40 command.spawn_token_fork_ms=0 command.system_time_ms=31 command.user_time_ms=13 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:37:49.070" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:39.301" grpc.time_ms=48.47 pid=906 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 236.029677] gitaly[906]: time="2026-11-13T10:37:39.361Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-13T10:37:49.130" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:39.361" grpc.time_ms=0.111 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 236.039730] gitaly[906]: time="2026-11-13T10:37:39.371Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=938 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-13T10:37:49.133" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:39.363" grpc.time_ms=7.504 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 236.092550] gitaly[906]: time="2026-11-13T10:37:39.424Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=341 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-11-13T10:38:09.020" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:39.420" grpc.time_ms=3.686 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 236.125063] gitaly[906]: time="2026-11-13T10:37:39.456Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-13T10:37:49.226" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:39.456" grpc.time_ms=0.2 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 236.131289] gitaly[906]: time="2026-11-13T10:37:39.463Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-13T10:37:49.227" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-11-13T10:37:39.457" grpc.time_ms=5 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 236.166476] gitaly[906]: time="2026-11-13T10:37:39.498Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:49.259" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:39.489" grpc.time_ms=8.265 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 236.179856] gitaly[906]: time="2026-11-13T10:37:39.511Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:49.272" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:39.503" grpc.time_ms=8.507 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 236.199481] gitaly[906]: time="2026-11-13T10:37:39.531Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-13T10:38:09.120" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:39.520" grpc.time_ms=10.382 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340' gitlab # [ 236.302671] nixbot[2602]: {"ts": "2026-11-13T10:37:39.633Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:37902 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 HTTP/1.1\" 200"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340', in 0.05 seconds) gitlab # [ 236.532653] gitaly[906]: time="2026-11-13T10:37:39.864Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-13T10:37:49.002" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:39.863" grpc.time_ms=0.962 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 236.554805] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:39.885Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 5feeafd1ab71e41915719113facb270437fd8a7e refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"2420aca0cac16d9f01c96596","created_at":"2026-11-13T10:36:55.589Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-d105d3f8f92a48a8","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:36:55.594Z","job_size_bytes":185,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-2420aca0cac16d9f01c96596: done: 0.683162 sec","job_status":"done","queue_duration_s":43.608099,"scheduling_latency_s":43.608112,"gitaly_calls":13,"gitaly_duration_s":0.171705,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.032691,"redis_read_bytes":2103,"redis_write_bytes":21748,"redis_cache_calls":15,"redis_cache_duration_s":0.002711,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.004807,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.00098,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7836,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003385,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.012886,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.007922,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.176,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.189,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.444329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000415416,"exclusive_lock_hold_duration_s":0.024639723,"duration_s":0.683162,"completed_at":"2026-11-13T10:37:39.885Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.053798,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 236.574226] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:39.886Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"27908a47a8c34b7076677d51","created_at":"2026-11-13T10:37:00.477Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-c0ccb197c5274ff0","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:00.484Z","job_size_bytes":65,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-27908a47a8c34b7076677d51: start","job_status":"start","queue_duration_s":39.402308,"scheduling_latency_s":39.402325} gitlab # [ 236.637538] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:39.936Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"27908a47a8c34b7076677d51","created_at":"2026-11-13T10:37:00.477Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-c0ccb197c5274ff0","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:00.484Z","job_size_bytes":65,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-27908a47a8c34b7076677d51: done: 0.049711 sec","job_status":"done","queue_duration_s":39.402308,"scheduling_latency_s":39.402325,"redis_calls":16,"redis_duration_s":0.002404,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001195,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034346,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000249473,"duration_s":0.049711,"completed_at":"2026-11-13T10:37:39.936Z","load_balancing_strategy":"primary","db_duration_s":0.010661,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 236.641704] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:39.937Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"93db4041930c96b6e989467a","created_at":"2026-11-13T10:37:00.485Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-c0ccb197c5274ff0","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:00.491Z","job_size_bytes":65,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-93db4041930c96b6e989467a: start","job_status":"start","queue_duration_s":39.446454,"scheduling_latency_s":39.446468} gitlab # [ 236.688489] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:39.996Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"93db4041930c96b6e989467a","created_at":"2026-11-13T10:37:00.485Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-c0ccb197c5274ff0","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:00.491Z","job_size_bytes":65,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-93db4041930c96b6e989467a: done: 0.059083 sec","job_status":"done","queue_duration_s":39.446454,"scheduling_latency_s":39.446468,"redis_calls":21,"redis_duration_s":0.00297,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.001252,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000796,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212597,"duration_s":0.059083,"completed_at":"2026-11-13T10:37:39.996Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015966,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.696212] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:39.997Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"86013c376836fda5d1771e76","created_at":"2026-11-13T10:37:00.585Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-c0ccb197c5274ff0","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:00.589Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-86013c376836fda5d1771e76: start","job_status":"start","queue_duration_s":39.40805,"scheduling_latency_s":39.408064} gitlab # [ 236.852586] gitaly[906]: time="2026-11-13T10:37:40.184Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:40.183" grpc.time_ms=0.722 pid=906 span.kind=server system=grpc gitlab # [ 236.905592] gitaly[906]: time="2026-11-13T10:37:40.237Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:40.236" grpc.time_ms=0.815 pid=906 span.kind=server system=grpc gitlab # [ 236.951345] gitaly[906]: time="2026-11-13T10:37:40.282Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:40.282" grpc.time_ms=0.307 pid=906 span.kind=server system=grpc gitlab # [ 236.975566] gitaly[906]: time="2026-11-13T10:37:40.307Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M diskcache=bab9018c-fa04-4201-95d3-105bba8b4341 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-13T10:38:10.045" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:40.146" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 236.984243] gitaly[906]: time="2026-11-13T10:37:40.316Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=153 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7533 command.oublock=0 command.real_time_ms=154 command.spawn_token_fork_ms=0 command.system_time_ms=87 command.user_time_ms=66 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-13T10:38:10.045" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:40.146" grpc.time_ms=161.384 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.088523] gitaly[906]: time="2026-11-13T10:37:40.420Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-13T10:38:10.018" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:40.419" grpc.time_ms=1.009 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.125247] gitaly[906]: time="2026-11-13T10:37:40.456Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PTBH7GNFVV1QVWG9TS0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:10.055" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:40.456" grpc.time_ms=0.66 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.132784] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:40.464Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"86013c376836fda5d1771e76","created_at":"2026-11-13T10:37:00.585Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-c0ccb197c5274ff0","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:00.589Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2485,"job_status":"rescheduled","message":"PipelineProcessWorker JID-86013c376836fda5d1771e76: rescheduled"} gitlab # [ 237.139409] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:40.470Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"86013c376836fda5d1771e76","created_at":"2026-11-13T10:37:00.585Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-c0ccb197c5274ff0","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:00.589Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-86013c376836fda5d1771e76: done: 0.473073 sec","job_status":"done","queue_duration_s":39.40805,"scheduling_latency_s":39.408064,"gitaly_calls":3,"gitaly_duration_s":0.178111,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.024967,"redis_read_bytes":460,"redis_write_bytes":29295,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000851,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001157,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.008834,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.004,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15582,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.005306,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000554,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000838,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002576,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.055,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.074,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.076,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.023,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.214576,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.438571383,"duration_s":0.473073,"completed_at":"2026-11-13T10:37:40.470Z","load_balancing_strategy":"primary","db_duration_s":0.06536,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 237.191127] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:40.472Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","5feeafd1ab71e41915719113facb270437fd8a7e bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"adb826e6fd4994c0ff7196c0","created_at":"2026-11-13T10:37:38.418Z","trace_propagation_headers":{"sentry-trace":"9b5c55734ded4ab5a209e411003e4a98-30b279719cba46da","baggage":"sentry-trace_id=9b5c55734ded4ab5a209e411003e4a98,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7QZ968R6WB2HBDDBP3611","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:38.421Z","job_size_bytes":185,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-adb826e6fd4994c0ff7196c0: start","job_status":"start","queue_duration_s":2.050727,"scheduling_latency_s":2.050744} gitlab # [ 237.191744] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:40.480Z","message":"changes: \"5feeafd1ab71e41915719113facb270437fd8a7e bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 refs/heads/master\\n\""} gitlab # [ 237.193587] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:40.517Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"5f86d067af4b2f9af4aecc66","created_at":"2026-11-13T10:37:40.515Z","trace_propagation_headers":{"sentry-trace":"9b5c55734ded4ab5a209e411003e4a98-a0be5b3fe69e4aa1","baggage":"sentry-trace_id=9b5c55734ded4ab5a209e411003e4a98,sentry-release=Unknown"},"correlation_id":"01M7A7QZ968R6WB2HBDDBP3611","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"80fc8864b849373cb7719531","job_size_bytes":46,"pid":2485,"job_status":"deduplicated","message":"ProjectCacheWorker JID-5f86d067af4b2f9af4aecc66: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 237.206893] gitaly[906]: time="2026-11-13T10:37:40.502Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:50.262" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:40.492" grpc.time_ms=9.55 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.218310] gitaly[906]: time="2026-11-13T10:37:40.514Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-13T10:37:50.274" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:40.505" grpc.time_ms=8.737 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.262086] gitaly[906]: time="2026-11-13T10:37:40.533Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-13T10:37:50.295" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:40.525" grpc.time_ms=7.542 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.279906] gitaly[906]: time="2026-11-13T10:37:40.611Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=68 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=68 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-11-13T10:38:10.136" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:40.537" grpc.time_ms=74.412 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.291698] gitaly[906]: time="2026-11-13T10:37:40.623Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-13T10:37:50.009" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-11-13T10:37:40.619" grpc.time_ms=3.863 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340' gitlab # [ 237.319144] gitaly[906]: time="2026-11-13T10:37:40.650Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:50.032" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:40.642" grpc.time_ms=8.475 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.338146] nixbot[2602]: {"ts": "2026-11-13T10:37:40.667Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:37908 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 HTTP/1.1\" 200"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340', in 0.03 seconds) gitlab # [ 237.385698] gitaly[906]: time="2026-11-13T10:37:40.717Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-13T10:37:50.105" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:40.716" grpc.time_ms=0.917 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.419987] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:40.751Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","5feeafd1ab71e41915719113facb270437fd8a7e bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"adb826e6fd4994c0ff7196c0","created_at":"2026-11-13T10:37:38.418Z","trace_propagation_headers":{"sentry-trace":"9b5c55734ded4ab5a209e411003e4a98-30b279719cba46da","baggage":"sentry-trace_id=9b5c55734ded4ab5a209e411003e4a98,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7QZ968R6WB2HBDDBP3611","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-13T10:37:38.421Z","job_size_bytes":185,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-adb826e6fd4994c0ff7196c0: done: 0.278877 sec","job_status":"done","queue_duration_s":2.050727,"scheduling_latency_s":2.050744,"gitaly_calls":7,"gitaly_duration_s":0.138922,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.036759,"redis_read_bytes":501,"redis_write_bytes":11941,"redis_cache_calls":9,"redis_cache_duration_s":0.001704,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000664,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5718,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002164,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.016666,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.015561,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.015,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.015,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.091325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00045313,"exclusive_lock_hold_duration_s":0.016205691,"duration_s":0.278877,"completed_at":"2026-11-13T10:37:40.751Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020737,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 237.464483] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:40.795Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"80fc8864b849373cb7719531","created_at":"2026-11-13T10:37:39.259Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-9f828bdd85854f5c","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:39.274Z","job_size_bytes":46,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-80fc8864b849373cb7719531: start","job_status":"start","queue_duration_s":1.520738,"scheduling_latency_s":1.520779} gitlab # [ 237.537848] gitaly[906]: time="2026-11-13T10:37:40.869Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-11-13T16:37:40.863" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:40.864" grpc.time_ms=4.784 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.559945] gitaly[906]: time="2026-11-13T10:37:40.891Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=655 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-13T10:38:10.479" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:40.879" grpc.time_ms=11.724 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.593495] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:40.923Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"80fc8864b849373cb7719531","created_at":"2026-11-13T10:37:39.259Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-9f828bdd85854f5c","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:39.274Z","job_size_bytes":46,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-80fc8864b849373cb7719531: done: 0.128336 sec","job_status":"done","queue_duration_s":1.520738,"scheduling_latency_s":1.520779,"gitaly_calls":2,"gitaly_duration_s":0.022513,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008246,"redis_read_bytes":69,"redis_write_bytes":4886,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001704,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000754,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2114,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002285,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.00273,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000773,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000372394,"duration_s":0.128336,"completed_at":"2026-11-13T10:37:40.923Z","load_balancing_strategy":"primary","db_duration_s":0.012428,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 237.597498] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:40.925Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0e6f56e466db906b8200e0cd","created_at":"2026-11-13T10:37:39.284Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-9f828bdd85854f5c","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:39.286Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0e6f56e466db906b8200e0cd: start","job_status":"start","queue_duration_s":1.638988,"scheduling_latency_s":1.639012} gitlab # [ 237.710060] gitaly[906]: time="2026-11-13T10:37:41.041Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT diskcache=9fd8e599-2aee-4b05-88f3-d3c4436e5429 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-13T16:37:41.013" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:41.013" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.710728] gitaly[906]: time="2026-11-13T10:37:41.042Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1294 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-13T16:37:41.013" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:41.013" grpc.time_ms=28.026 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.761863] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:41.092Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0e6f56e466db906b8200e0cd","created_at":"2026-11-13T10:37:39.284Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-9f828bdd85854f5c","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:39.286Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0e6f56e466db906b8200e0cd: done: 0.167303 sec","job_status":"done","queue_duration_s":1.638988,"scheduling_latency_s":1.639012,"gitaly_calls":1,"gitaly_duration_s":0.031371,"redis_calls":15,"redis_duration_s":0.005271,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000601,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002013,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000829,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001828,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.013,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.026,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.077647298,"duration_s":0.167303,"completed_at":"2026-11-13T10:37:41.092Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020627,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.765319] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:41.095Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"154b7355854f498276216344","created_at":"2026-11-13T10:37:39.357Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-9f828bdd85854f5c","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:39.359Z","job_size_bytes":23,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-154b7355854f498276216344: start","job_status":"start","queue_duration_s":1.735926,"scheduling_latency_s":1.735953} gitlab # [ 237.795989] gitaly[906]: time="2026-11-13T10:37:41.127Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=655 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-13T10:38:11.021" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:41.122" grpc.time_ms=5.396 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.802636] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:41.134Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"154b7355854f498276216344","created_at":"2026-11-13T10:37:39.357Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-9f828bdd85854f5c","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:39.359Z","job_size_bytes":23,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-154b7355854f498276216344: done: 0.038887 sec","job_status":"done","queue_duration_s":1.735926,"scheduling_latency_s":1.735953,"gitaly_calls":1,"gitaly_duration_s":0.007435,"redis_calls":16,"redis_duration_s":0.003766,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000184,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001238,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001511,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000833,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038887,"completed_at":"2026-11-13T10:37:41.134Z","load_balancing_strategy":"primary","db_duration_s":0.003166,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 237.835728] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:41.135Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"f47aeca62aac29449709e4af","created_at":"2026-11-13T10:37:39.758Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-9f828bdd85854f5c","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1e150a66297a5a9bb004bd70e00b03a86fd855977488fcffea2719970f8bc41","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:39.762Z","job_size_bytes":467,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-f47aeca62aac29449709e4af: start","job_status":"start","queue_duration_s":1.372489,"scheduling_latency_s":1.372505} gitlab # [ 237.885609] gitaly[906]: time="2026-11-13T10:37:41.152Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:11.051" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:41.152" grpc.time_ms=0.373 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.886479] gitaly[906]: time="2026-11-13T10:37:41.159Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-13T10:37:51.003" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:41.153" grpc.time_ms=5.523 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.898184] gitaly[906]: time="2026-11-13T10:37:41.229Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-13T10:38:11.128" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:41.229" grpc.time_ms=0.695 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.902697] gitaly[906]: time="2026-11-13T10:37:41.234Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=47 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-13T10:38:11.132" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:41.232" grpc.time_ms=1.632 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 237.956496] gitaly[906]: time="2026-11-13T10:37:41.287Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:51.132" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:41.282" grpc.time_ms=5.499 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340' gitlab # [ 238.363316] gitaly[906]: time="2026-11-13T10:37:41.694Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=1 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=968 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-13T10:38:11.575" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:41.676" grpc.time_ms=18.618 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 238.366605] nixbot[2602]: {"ts": "2026-11-13T10:37:41.697Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:37922 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 HTTP/1.1\" 200"} gitlab # [ 238.378238] gitaly[906]: time="2026-11-13T10:37:41.709Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7PNAYRC86EBJ42CEPW6DT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-13T10:38:11.599" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:41.699" grpc.time_ms=9.959 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 238.401509] sidekiq-cluster[2485]: {"severity":"WARN","time":"2026-11-13T10:37:41.733Z","class":"CreatePipelineWorker","correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"5feeafd1ab71e41915719113facb270437fd8a7e","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"5feeafd1ab71e41915719113facb270437fd8a7e","ref":"refs/heads/master"},"retry":0} gitlab # [ 238.407360] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:41.738Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"f47aeca62aac29449709e4af","created_at":"2026-11-13T10:37:39.758Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-9f828bdd85854f5c","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1e150a66297a5a9bb004bd70e00b03a86fd855977488fcffea2719970f8bc41","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:39.762Z","job_size_bytes":467,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-f47aeca62aac29449709e4af: done: 0.603532 sec","job_status":"done","queue_duration_s":1.372489,"scheduling_latency_s":1.372505,"gitaly_calls":7,"gitaly_duration_s":0.057417,"redis_calls":51,"redis_duration_s":0.007255,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000573,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.003496,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000947,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001508,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000731,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.507007,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000230197,"duration_s":0.603532,"completed_at":"2026-11-13T10:37:41.738Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029884,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340', in 0.14 seconds) gitlab # [ 238.437040] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:41.740Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"d45d6da7-e3be-4264-a6ad-01c000e06812\"}"],"class":"WebHookWorker","jid":"784807b5f7e5acfd6afa5e09","created_at":"2026-11-13T10:37:39.844Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-9f828bdd85854f5c","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:97a4fbb42726f496151b7bfeeaccf25636cc608ad85c3ae48f23e9ddf5ad8f12","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:39.848Z","job_size_bytes":1751,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-784807b5f7e5acfd6afa5e09: start","job_status":"start","queue_duration_s":1.891708,"scheduling_latency_s":1.891724} gitlab # [ 238.548449] nixbot[2602]: {"ts": "2026-11-13T10:37:41.880Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "1e2caa17-14eb-4764-aad0-fef5117e1a45", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "5feeafd1ab71e41915719113facb270437fd8a7e", "pr": null} gitlab # [ 238.551840] nixbot[2602]: {"ts": "2026-11-13T10:37:41.883Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:37936 - \"POST /webhooks/gitlab HTTP/1.1\" 202"} gitlab # [ 238.571256] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:41.902Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"d45d6da7-e3be-4264-a6ad-01c000e06812\"}"],"class":"WebHookWorker","jid":"784807b5f7e5acfd6afa5e09","created_at":"2026-11-13T10:37:39.844Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-9f828bdd85854f5c","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:97a4fbb42726f496151b7bfeeaccf25636cc608ad85c3ae48f23e9ddf5ad8f12","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:39.848Z","job_size_bytes":1751,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-784807b5f7e5acfd6afa5e09: done: 0.162359 sec","job_status":"done","queue_duration_s":1.891708,"scheduling_latency_s":1.891724,"redis_calls":19,"redis_duration_s":0.002221,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000555,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000196,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000958,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000512,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.04124043699999902,"cpu_s":0.057337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000234387,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.162359,"completed_at":"2026-11-13T10:37:41.902Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003918,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.607675] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:41.903Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"a71162c2f587e8dfccfdfc8c","created_at":"2026-11-13T10:37:40.127Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.130Z","job_size_bytes":23,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-a71162c2f587e8dfccfdfc8c: start","job_status":"start","queue_duration_s":1.77335,"scheduling_latency_s":1.773376} gitlab # [ 238.638949] gitaly[906]: time="2026-11-13T10:37:41.970Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7R34NKXQ1CX3AFYNE5XH5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:41.964" grpc.time_ms=5.716 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 238.640518] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:41 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 62 gitlab # [ 238.688024] gitaly[906]: time="2026-11-13T10:37:42.019Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R36NMX7X7G562NNANAN0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:42.013" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 238.688534] gitaly[906]: time="2026-11-13T10:37:42.019Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R36NMX7X7G562NNANAN0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:42.013" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 238.689221] gitaly[906]: time="2026-11-13T10:37:42.020Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R36NMX7X7G562NNANAN0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:42.013" grpc.time_ms=6.213 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 238.720814] gitlab-workhorse[827]: time="2026-11-13T10:37:42Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 238.721120] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:42 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 79 gitlab # [ 238.727873] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.059Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"a71162c2f587e8dfccfdfc8c","created_at":"2026-11-13T10:37:40.127Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.130Z","job_size_bytes":23,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-a71162c2f587e8dfccfdfc8c: done: 0.155717 sec","job_status":"done","queue_duration_s":1.77335,"scheduling_latency_s":1.773376,"redis_calls":10,"redis_duration_s":0.037934999999999997,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001686,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003926,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.032323,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.058,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.054,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000323226,"duration_s":0.155717,"completed_at":"2026-11-13T10:37:42.059Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.060665,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 238.766897] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.098Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e76bf5c6486a4a172a7b2006","created_at":"2026-11-13T10:37:40.322Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.328Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e76bf5c6486a4a172a7b2006: start","job_status":"start","queue_duration_s":1.76988,"scheduling_latency_s":1.769901} gitlab # [ 238.792104] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.123Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e76bf5c6486a4a172a7b2006","created_at":"2026-11-13T10:37:40.322Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.328Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e76bf5c6486a4a172a7b2006: done: 0.025165 sec","job_status":"done","queue_duration_s":1.76988,"scheduling_latency_s":1.769901,"redis_calls":10,"redis_duration_s":0.002101,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000412,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001288,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000401,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000268191,"duration_s":0.025165,"completed_at":"2026-11-13T10:37:42.123Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005182,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.794322] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.125Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f4429bd619a5ac0267bb444a","created_at":"2026-11-13T10:37:40.344Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.350Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f4429bd619a5ac0267bb444a: start","job_status":"start","queue_duration_s":1.774756,"scheduling_latency_s":1.77477} gitlab # [ 238.822591] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.153Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f4429bd619a5ac0267bb444a","created_at":"2026-11-13T10:37:40.344Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.350Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f4429bd619a5ac0267bb444a: done: 0.028075 sec","job_status":"done","queue_duration_s":1.774756,"scheduling_latency_s":1.77477,"redis_calls":10,"redis_duration_s":0.001398,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000343,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000219301,"duration_s":0.028075,"completed_at":"2026-11-13T10:37:42.153Z","load_balancing_strategy":"primary","db_duration_s":0.004613,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.845392] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.155Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"243134f7fdb0281fe9bc5741","created_at":"2026-11-13T10:37:40.366Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.370Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-243134f7fdb0281fe9bc5741: start","job_status":"start","queue_duration_s":1.784204,"scheduling_latency_s":1.78422} gitlab # [ 238.858699] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.177Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"243134f7fdb0281fe9bc5741","created_at":"2026-11-13T10:37:40.366Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.370Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-243134f7fdb0281fe9bc5741: done: 0.022319 sec","job_status":"done","queue_duration_s":1.784204,"scheduling_latency_s":1.78422,"redis_calls":13,"redis_duration_s":0.001489,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000228,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000682,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000344,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000235,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000214273,"duration_s":0.022319,"completed_at":"2026-11-13T10:37:42.177Z","load_balancing_strategy":"primary","db_duration_s":0.002803,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.909627] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.178Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d193ce80d15cf19cd5220d0d","created_at":"2026-11-13T10:37:40.371Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.376Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d193ce80d15cf19cd5220d0d: start","job_status":"start","queue_duration_s":1.801901,"scheduling_latency_s":1.801914} gitlab # [ 238.911174] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.197Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d193ce80d15cf19cd5220d0d","created_at":"2026-11-13T10:37:40.371Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.376Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d193ce80d15cf19cd5220d0d: done: 0.01903 sec","job_status":"done","queue_duration_s":1.801901,"scheduling_latency_s":1.801914,"redis_calls":10,"redis_duration_s":0.001177,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000232,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000719,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000226,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000185778,"duration_s":0.01903,"completed_at":"2026-11-13T10:37:42.197Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002477,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.949702] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.198Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"73160c798a961da2df25008a","created_at":"2026-11-13T10:37:40.377Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.382Z","job_size_bytes":19,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-73160c798a961da2df25008a: start","job_status":"start","queue_duration_s":1.815574,"scheduling_latency_s":1.815586} gitlab # [ 238.987372] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.214Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"73160c798a961da2df25008a","created_at":"2026-11-13T10:37:40.377Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.382Z","job_size_bytes":19,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-73160c798a961da2df25008a: done: 0.016747 sec","job_status":"done","queue_duration_s":1.815574,"scheduling_latency_s":1.815586,"redis_calls":11,"redis_duration_s":0.0013289999999999999,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000852,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000228,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010565,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000181587,"duration_s":0.016747,"completed_at":"2026-11-13T10:37:42.214Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002034,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.001073] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.215Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0cd3e00dfa82d47c512f7584","created_at":"2026-11-13T10:37:40.385Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.388Z","job_size_bytes":66,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0cd3e00dfa82d47c512f7584: start","job_status":"start","queue_duration_s":1.827636,"scheduling_latency_s":1.827649} gitlab # [ 239.016254] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.297Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0cd3e00dfa82d47c512f7584","created_at":"2026-11-13T10:37:40.385Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.388Z","job_size_bytes":66,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0cd3e00dfa82d47c512f7584: done: 0.0817 sec","job_status":"done","queue_duration_s":1.827636,"scheduling_latency_s":1.827649,"redis_calls":10,"redis_duration_s":0.001268,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000727,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000238,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.064,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.062,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000198908,"duration_s":0.0817,"completed_at":"2026-11-13T10:37:42.297Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.064555,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.066684] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.298Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"71c5c8d947f9cbca35c0d87d","created_at":"2026-11-13T10:37:40.388Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.391Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-71c5c8d947f9cbca35c0d87d: start","job_status":"start","queue_duration_s":1.907077,"scheduling_latency_s":1.90709} gitlab # [ 239.069293] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.321Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"71c5c8d947f9cbca35c0d87d","created_at":"2026-11-13T10:37:40.388Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.391Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-71c5c8d947f9cbca35c0d87d: done: 0.022981 sec","job_status":"done","queue_duration_s":1.907077,"scheduling_latency_s":1.90709,"redis_calls":10,"redis_duration_s":0.001338,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000232,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000228,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01603,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000195835,"duration_s":0.022981,"completed_at":"2026-11-13T10:37:42.321Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003156,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.107885] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.322Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"95536a05e4218fc0fca0ef93","created_at":"2026-11-13T10:37:40.395Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.398Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-95536a05e4218fc0fca0ef93: start","job_status":"start","queue_duration_s":1.924334,"scheduling_latency_s":1.924347} gitlab # [ 239.145232] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.343Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"95536a05e4218fc0fca0ef93","created_at":"2026-11-13T10:37:40.395Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.398Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-95536a05e4218fc0fca0ef93: done: 0.020845 sec","job_status":"done","queue_duration_s":1.924334,"scheduling_latency_s":1.924347,"redis_calls":10,"redis_duration_s":0.001161,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000247,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000686,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000228,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000183822,"duration_s":0.020845,"completed_at":"2026-11-13T10:37:42.343Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005249,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.160026] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.344Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a08fd4365ff508b5b123db45","created_at":"2026-11-13T10:37:40.398Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.400Z","job_size_bytes":6,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a08fd4365ff508b5b123db45: start","job_status":"start","queue_duration_s":1.943218,"scheduling_latency_s":1.94323} gitlab # [ 239.174152] gitaly[906]: time="2026-11-13T10:37:42.462Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 diskcache=cd48e1c0-246d-4076-96da-2b65ba141f16 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-13T16:37:42.453" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:42.454" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 239.182516] gitaly[906]: time="2026-11-13T10:37:42.462Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=314 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-13T16:37:42.453" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:42.454" grpc.time_ms=8.102 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 239.223726] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.356Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a08fd4365ff508b5b123db45","created_at":"2026-11-13T10:37:40.398Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.400Z","job_size_bytes":6,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a08fd4365ff508b5b123db45: done: 0.012425 sec","job_status":"done","queue_duration_s":1.943218,"scheduling_latency_s":1.94323,"redis_calls":10,"redis_duration_s":0.001173,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000228,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000697,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00018941,"duration_s":0.012425,"completed_at":"2026-11-13T10:37:42.356Z","load_balancing_strategy":"primary","db_duration_s":0.001151,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.231020] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.357Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"a6f2a15d2349d97a0d5d2c1f","created_at":"2026-11-13T10:37:40.401Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.403Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a6f2a15d2349d97a0d5d2c1f: start","job_status":"start","queue_duration_s":1.953625,"scheduling_latency_s":1.953637} gitlab # [ 239.244467] gitaly[906]: time="2026-11-13T10:37:42.486Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:12.385" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:42.485" grpc.time_ms=0.965 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 239.255088] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.376Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"a6f2a15d2349d97a0d5d2c1f","created_at":"2026-11-13T10:37:40.401Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.403Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a6f2a15d2349d97a0d5d2c1f: done: 0.018984 sec","job_status":"done","queue_duration_s":1.953625,"scheduling_latency_s":1.953637,"redis_calls":10,"redis_duration_s":0.001152,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000222,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000706,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000224,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000190807,"duration_s":0.018984,"completed_at":"2026-11-13T10:37:42.376Z","load_balancing_strategy":"primary","db_duration_s":0.003269,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.302133] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.377Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"dd1be7d2fc46ea63c5214160","created_at":"2026-11-13T10:37:40.408Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.410Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-dd1be7d2fc46ea63c5214160: start","job_status":"start","queue_duration_s":1.966617,"scheduling_latency_s":1.96663} gitlab # [ 239.305944] gitaly[906]: time="2026-11-13T10:37:42.496Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-13T10:37:52.008" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:42.488" grpc.time_ms=7.828 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 239.317548] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.423Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"dd1be7d2fc46ea63c5214160","created_at":"2026-11-13T10:37:40.408Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.410Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-dd1be7d2fc46ea63c5214160: done: 0.046765 sec","job_status":"done","queue_duration_s":1.966617,"scheduling_latency_s":1.96663,"redis_calls":14,"redis_duration_s":0.002974,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00023,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001772,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000972,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000279365,"duration_s":0.046765,"completed_at":"2026-11-13T10:37:42.423Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.380263] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.425Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"7378bc6e8dafc43b2a4c8e84","created_at":"2026-11-13T10:37:40.464Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.466Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7378bc6e8dafc43b2a4c8e84: start","job_status":"start","queue_duration_s":1.958974,"scheduling_latency_s":1.958989} gitlab # [ 239.380944] gitaly[906]: time="2026-11-13T10:37:42.517Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-13T10:38:12.416" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:42.516" grpc.time_ms=0.587 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 239.381515] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.441Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"7378bc6e8dafc43b2a4c8e84","created_at":"2026-11-13T10:37:40.464Z","trace_propagation_headers":{"sentry-trace":"a841c79b9e2a4b68a28afb25f5dd09cc-df2e1f21d5274b5c","baggage":"sentry-trace_id=a841c79b9e2a4b68a28afb25f5dd09cc,sentry-release=Unknown"},"correlation_id":"01M7A7PTBH7GNFVV1QVWG9TS0M","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.466Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7378bc6e8dafc43b2a4c8e84: done: 0.016961 sec","job_status":"done","queue_duration_s":1.958974,"scheduling_latency_s":1.958989,"redis_calls":8,"redis_duration_s":0.001312,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00101,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000302,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016961,"completed_at":"2026-11-13T10:37:42.441Z","load_balancing_strategy":"primary","db_duration_s":0.003286,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 239.415689] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.442Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"660d41b4d3f13e66ed9c9cbe","created_at":"2026-11-13T10:37:40.520Z","trace_propagation_headers":{"sentry-trace":"9b5c55734ded4ab5a209e411003e4a98-a0be5b3fe69e4aa1","baggage":"sentry-trace_id=9b5c55734ded4ab5a209e411003e4a98,sentry-release=Unknown"},"correlation_id":"01M7A7QZ968R6WB2HBDDBP3611","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"0e6f56e466db906b8200e0cd","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.522Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-660d41b4d3f13e66ed9c9cbe: start","job_status":"start","queue_duration_s":1.920353,"scheduling_latency_s":1.920366} gitlab # [ 239.458254] gitaly[906]: time="2026-11-13T10:37:42.522Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=323 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-13T10:38:12.419" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:42.519" grpc.time_ms=2.696 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340' gitlab # [ 239.458915] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.471Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"660d41b4d3f13e66ed9c9cbe","created_at":"2026-11-13T10:37:40.520Z","trace_propagation_headers":{"sentry-trace":"9b5c55734ded4ab5a209e411003e4a98-a0be5b3fe69e4aa1","baggage":"sentry-trace_id=9b5c55734ded4ab5a209e411003e4a98,sentry-release=Unknown"},"correlation_id":"01M7A7QZ968R6WB2HBDDBP3611","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"0e6f56e466db906b8200e0cd","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.522Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-660d41b4d3f13e66ed9c9cbe: done: 0.028578 sec","job_status":"done","queue_duration_s":1.920353,"scheduling_latency_s":1.920366,"gitaly_calls":1,"gitaly_duration_s":0.009824,"redis_calls":12,"redis_duration_s":0.0021409999999999997,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00049,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00077,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.017905907,"duration_s":0.028578,"completed_at":"2026-11-13T10:37:42.471Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003513,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.482991] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.472Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"eb07e61f67937b139097df70","created_at":"2026-11-13T10:37:40.695Z","trace_propagation_headers":{"sentry-trace":"9b5c55734ded4ab5a209e411003e4a98-a0be5b3fe69e4aa1","baggage":"sentry-trace_id=9b5c55734ded4ab5a209e411003e4a98,sentry-release=Unknown"},"correlation_id":"01M7A7QZ968R6WB2HBDDBP3611","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:33c102c95956662d3ed237cb06553ea8ef20ffa8bcc3f667969f742683a10694","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.697Z","job_size_bytes":467,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-eb07e61f67937b139097df70: start","job_status":"start","queue_duration_s":1.775348,"scheduling_latency_s":1.77536} gitlab # [ 239.496908] gitaly[906]: time="2026-11-13T10:37:42.548Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:52.061" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:42.541" grpc.time_ms=6.308 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 239.550925] gitaly[906]: time="2026-11-13T10:37:42.882Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R396REX9X0M6246ZXKYX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:42.124" pid=906 remote_ip="2001:db8:1::1" request_sha=e77d2e393035e4b3c56bd6b44333eb2390f7a228 response_bytes=337 span.kind=server system=grpc gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340', in 0.08 seconds) gitlab # [ 239.551323] gitaly[906]: time="2026-11-13T10:37:42.882Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=946176 command.minflt=1199 command.oublock=0 command.real_time_ms=755 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R396REX9X0M6246ZXKYX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:42.124" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 239.552325] gitaly[906]: time="2026-11-13T10:37:42.884Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=946176 command.minflt=1199 command.oublock=0 command.real_time_ms=755 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R396REX9X0M6246ZXKYX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:42.124" grpc.time_ms=759.466 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 239.555063] gitlab-workhorse[827]: time="2026-11-13T10:37:42Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=337 gitlab # [ 239.557541] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:42 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 830 gitlab # [ 239.560880] nixbot[2602]: {"ts": "2026-11-13T10:37:42.892Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:37938 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 HTTP/1.1\" 200"} gitlab # [ 239.579556] nixbot[2602]: {"ts": "2026-11-13T10:37:42.911Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 239.580683] nixbot[2602]: {"ts": "2026-11-13T10:37:42.912Z", "level": "info", "logger": "nixbot.after_build", "msg": "reused build: replaying effect statuses", "build_id": 1, "sha": "5feeafd1ab71e41915719113facb270437fd8a7e"} gitlab # [ 239.610873] gitaly[906]: time="2026-11-13T10:37:42.942Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=962 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-13T10:38:12.834" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:42.934" grpc.time_ms=7.771 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 239.620596] gitaly[906]: time="2026-11-13T10:37:42.952Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7QZ968R6WB2HBDDBP3611 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-13T10:38:12.845" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:42.945" grpc.time_ms=6.903 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 239.624550] gitaly[906]: time="2026-11-13T10:37:42.956Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:12.855" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:42.955" grpc.time_ms=0.32 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 239.630045] sidekiq-cluster[2485]: {"severity":"WARN","time":"2026-11-13T10:37:42.961Z","class":"CreatePipelineWorker","correlation_id":"01M7A7QZ968R6WB2HBDDBP3611","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"5feeafd1ab71e41915719113facb270437fd8a7e","after":"bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340","ref":"refs/heads/master"},"retry":0} gitlab # [ 239.633370] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.964Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"eb07e61f67937b139097df70","created_at":"2026-11-13T10:37:40.695Z","trace_propagation_headers":{"sentry-trace":"9b5c55734ded4ab5a209e411003e4a98-a0be5b3fe69e4aa1","baggage":"sentry-trace_id=9b5c55734ded4ab5a209e411003e4a98,sentry-release=Unknown"},"correlation_id":"01M7A7QZ968R6WB2HBDDBP3611","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:33c102c95956662d3ed237cb06553ea8ef20ffa8bcc3f667969f742683a10694","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.697Z","job_size_bytes":467,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-eb07e61f67937b139097df70: done: 0.492347 sec","job_status":"done","queue_duration_s":1.775348,"scheduling_latency_s":1.77536,"gitaly_calls":7,"gitaly_duration_s":0.044738,"redis_calls":33,"redis_duration_s":0.007287,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000484,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.005442,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000583,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.410965,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.492347,"completed_at":"2026-11-13T10:37:42.964Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015368,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 239.655544] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.965Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"5c98db06-6b55-435f-b946-d6b7429dd7a3\"}"],"class":"WebHookWorker","jid":"e5e1f99aad647ac2f016654e","created_at":"2026-11-13T10:37:40.706Z","trace_propagation_headers":{"sentry-trace":"9b5c55734ded4ab5a209e411003e4a98-a0be5b3fe69e4aa1","baggage":"sentry-trace_id=9b5c55734ded4ab5a209e411003e4a98,sentry-release=Unknown"},"correlation_id":"01M7A7QZ968R6WB2HBDDBP3611","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:df87478c1469e824b75a7f5726721c0fe04619c784c571eba57b5b7adb9f8411","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.707Z","job_size_bytes":1749,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-e5e1f99aad647ac2f016654e: start","job_status":"start","queue_duration_s":2.258185,"scheduling_latency_s":2.258201} gitlab # [ 239.695314] nixbot[2602]: {"ts": "2026-11-13T10:37:42.981Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "69a426e5-bfab-4db3-9882-9670f726f22e", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340", "pr": null} gitlab # [ 239.695929] nixbot[2602]: {"ts": "2026-11-13T10:37:42.983Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:37954 - \"POST /webhooks/gitlab HTTP/1.1\" 202"} gitlab # [ 239.696574] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.993Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"5c98db06-6b55-435f-b946-d6b7429dd7a3\"}"],"class":"WebHookWorker","jid":"e5e1f99aad647ac2f016654e","created_at":"2026-11-13T10:37:40.706Z","trace_propagation_headers":{"sentry-trace":"9b5c55734ded4ab5a209e411003e4a98-a0be5b3fe69e4aa1","baggage":"sentry-trace_id=9b5c55734ded4ab5a209e411003e4a98,sentry-release=Unknown"},"correlation_id":"01M7A7QZ968R6WB2HBDDBP3611","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:df87478c1469e824b75a7f5726721c0fe04619c784c571eba57b5b7adb9f8411","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.707Z","job_size_bytes":1749,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-e5e1f99aad647ac2f016654e: done: 0.027573 sec","job_status":"done","queue_duration_s":2.258185,"scheduling_latency_s":2.258201,"redis_calls":14,"redis_duration_s":0.001833,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000279,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000616,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.005175519000005124,"cpu_s":0.01662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.027573,"completed_at":"2026-11-13T10:37:42.993Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.715956] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:42.994Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"c2f3edd982eaecb43b911ba8","created_at":"2026-11-13T10:37:40.903Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-a083d12bc1434a72","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.909Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-c2f3edd982eaecb43b911ba8: start","job_status":"start","queue_duration_s":2.084763,"scheduling_latency_s":2.084781} gitlab # [ 239.729685] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.024Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"c2f3edd982eaecb43b911ba8","created_at":"2026-11-13T10:37:40.903Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-a083d12bc1434a72","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:40.909Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-c2f3edd982eaecb43b911ba8: done: 0.029642 sec","job_status":"done","queue_duration_s":2.084763,"scheduling_latency_s":2.084781,"redis_calls":19,"redis_duration_s":0.002498,"redis_read_bytes":21,"redis_write_bytes":4290,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000638,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000291,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2261,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001192,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000377,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0045232,"duration_s":0.029642,"completed_at":"2026-11-13T10:37:43.024Z","load_balancing_strategy":"primary","db_duration_s":0.004118,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.805597] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.025Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","784807b5f7e5acfd6afa5e09"],"class":"WebHooks::LogExecutionWorker","jid":"24b831f75a7e240262c217d2","created_at":"2026-11-13T10:37:41.893Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-d209a43025194f64","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5fb836cfad51483a1842cfbfe37471a8bf8020b5e28aa58dc67a4ba9f0137174","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:41.897Z","job_size_bytes":2593,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-24b831f75a7e240262c217d2: start","job_status":"start","queue_duration_s":1.12815,"scheduling_latency_s":1.128162} gitlab # [ 239.806287] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.061Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","784807b5f7e5acfd6afa5e09"],"class":"WebHooks::LogExecutionWorker","jid":"24b831f75a7e240262c217d2","created_at":"2026-11-13T10:37:41.893Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-d209a43025194f64","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5fb836cfad51483a1842cfbfe37471a8bf8020b5e28aa58dc67a4ba9f0137174","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:41.897Z","job_size_bytes":2593,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-24b831f75a7e240262c217d2: done: 0.036124 sec","job_status":"done","queue_duration_s":1.12815,"scheduling_latency_s":1.128162,"redis_calls":10,"redis_duration_s":0.003346,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000269,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002246,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000831,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000265118,"duration_s":0.036124,"completed_at":"2026-11-13T10:37:43.061Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004936,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.821974] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.064Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","e5e1f99aad647ac2f016654e"],"class":"WebHooks::LogExecutionWorker","jid":"c9982560d4909182e6436579","created_at":"2026-11-13T10:37:42.985Z","trace_propagation_headers":{"sentry-trace":"9b5c55734ded4ab5a209e411003e4a98-0db1e95d806a46c6","baggage":"sentry-trace_id=9b5c55734ded4ab5a209e411003e4a98,sentry-release=Unknown"},"correlation_id":"01M7A7QZ968R6WB2HBDDBP3611","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0818fd617a18efc0d05325de8c8e6f304cd1ea08f55af4a5931dc5fed0b112ad","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:42.988Z","job_size_bytes":2592,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c9982560d4909182e6436579: start","job_status":"start","queue_duration_s":0.075936,"scheduling_latency_s":0.07595} gitlab # [ 239.837155] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:43 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 54 gitlab # [ 239.838602] gitlab-workhorse[827]: time="2026-11-13T10:37:43Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 239.840365] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:43 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 42 gitlab # [ 239.841728] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5feeafd1ab71e41915719113facb270437fd8a7e HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 221 gitlab # [ 239.844183] gitaly[906]: time="2026-11-13T10:37:43.086Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7R47R08SHEZA746KSF8HM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=196 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.079" grpc.time_ms=6.808 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 239.884204] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.097Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","e5e1f99aad647ac2f016654e"],"class":"WebHooks::LogExecutionWorker","jid":"c9982560d4909182e6436579","created_at":"2026-11-13T10:37:42.985Z","trace_propagation_headers":{"sentry-trace":"9b5c55734ded4ab5a209e411003e4a98-0db1e95d806a46c6","baggage":"sentry-trace_id=9b5c55734ded4ab5a209e411003e4a98,sentry-release=Unknown"},"correlation_id":"01M7A7QZ968R6WB2HBDDBP3611","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0818fd617a18efc0d05325de8c8e6f304cd1ea08f55af4a5931dc5fed0b112ad","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:42.988Z","job_size_bytes":2592,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c9982560d4909182e6436579: done: 0.033181 sec","job_status":"done","queue_duration_s":0.075936,"scheduling_latency_s":0.07595,"redis_calls":7,"redis_duration_s":0.008453,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007394,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001059,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033181,"completed_at":"2026-11-13T10:37:43.097Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005532,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.892519] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.100Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"99e2cfabde2d3a1eb874a475","created_at":"2026-11-13T10:37:43.015Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-ffeafd9795a14be0","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.018Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-99e2cfabde2d3a1eb874a475: start","job_status":"start","queue_duration_s":0.082087,"scheduling_latency_s":0.082142} gitlab # [ 239.907306] gitaly[906]: time="2026-11-13T10:37:43.105Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:53.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:43.099" grpc.time_ms=5.909 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 239.918935] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.165Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"99e2cfabde2d3a1eb874a475","created_at":"2026-11-13T10:37:43.015Z","trace_propagation_headers":{"sentry-trace":"ea1a9ccf1af243a6a8366dfcaa3461d7-ffeafd9795a14be0","baggage":"sentry-trace_id=ea1a9ccf1af243a6a8366dfcaa3461d7,sentry-release=Unknown"},"correlation_id":"01M7A7PNAYRC86EBJ42CEPW6DT","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.018Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-99e2cfabde2d3a1eb874a475: done: 0.064849 sec","job_status":"done","queue_duration_s":0.082087,"scheduling_latency_s":0.082142,"redis_calls":11,"redis_duration_s":0.002133,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001439,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000413,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028607,"duration_s":0.064849,"completed_at":"2026-11-13T10:37:43.165Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014683,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.963377] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.167Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"23f19a74c5b6d519191a22aa","created_at":"2026-11-13T10:37:43.130Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-9bfe0c326c7b4bd6","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.132Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-23f19a74c5b6d519191a22aa: start","job_status":"start","queue_duration_s":0.034343,"scheduling_latency_s":0.034358} gitlab # [ 239.971566] gitaly[906]: time="2026-11-13T10:37:43.129Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R49GFSJP01CJV2ZX31EW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.123" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 239.983451] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5feeafd1ab71e41915719113facb270437fd8a7e HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 123 gitlab # [ 239.984136] gitaly[906]: time="2026-11-13T10:37:43.129Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R49GFSJP01CJV2ZX31EW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.123" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 239.994254] gitaly[906]: time="2026-11-13T10:37:43.130Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R49GFSJP01CJV2ZX31EW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.123" grpc.time_ms=6.333 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 240.042253] gitaly[906]: time="2026-11-13T10:37:43.185Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R4BFAS4S8K0WSJV3A4KZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:13.084" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:43.185" grpc.time_ms=0.548 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.043687] gitaly[906]: time="2026-11-13T10:37:43.243Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=19 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7R4BFAS4S8K0WSJV3A4KZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:53.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:43.222" grpc.time_ms=21.351 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.046634] gitaly[906]: time="2026-11-13T10:37:43.292Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:43.292" grpc.time_ms=0.286 pid=906 span.kind=server system=grpc gitlab # [ 240.047407] gitaly[906]: time="2026-11-13T10:37:43.325Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R4FJKY87TZSK33JWTHAM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:13.224" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:43.324" grpc.time_ms=0.405 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.048585] gitaly[906]: time="2026-11-13T10:37:43.341Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:43.338" grpc.time_ms=0.638 pid=906 span.kind=server system=grpc gitlab # [ 240.056938] gitaly[906]: time="2026-11-13T10:37:43.355Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7R4FJKY87TZSK33JWTHAM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:53.259" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:43.350" grpc.time_ms=5.572 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.068719] gitaly[906]: time="2026-11-13T10:37:43.363Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:43.361" grpc.time_ms=2.233 pid=906 span.kind=server system=grpc gitlab # [ 240.077515] gitaly[906]: time="2026-11-13T10:37:43.375Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST diskcache=d5a698c2-829f-4ff2-b269-946c9a27323a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-13T10:38:13.166" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:43.266" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.120879] gitaly[906]: time="2026-11-13T10:37:43.375Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=59 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6886 command.oublock=0 command.real_time_ms=100 command.spawn_token_fork_ms=1 command.system_time_ms=41 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-13T10:38:13.166" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:43.266" grpc.time_ms=108.49 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.121562] gitaly[906]: time="2026-11-13T10:37:43.425Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-13T10:38:13.324" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:43.424" grpc.time_ms=0.508 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.122302] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/5feeafd1ab71e41915719113facb270437fd8a7e HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 134 gitlab # [ 240.122952] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:43 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 134 gitlab # [ 240.123512] gitaly[906]: time="2026-11-13T10:37:43.445Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M7A7R4GEPDTVJ21E6VH37WDD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.437" grpc.time_ms=6.333 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 240.124253] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.454Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"f9d7feab586433a9b80a09de","created_at":"2026-11-13T10:37:43.452Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"23f19a74c5b6d519191a22aa","job_size_bytes":3,"pid":2485,"job_status":"deduplicated","message":"PipelineProcessWorker JID-f9d7feab586433a9b80a09de: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 240.136716] gitaly[906]: time="2026-11-13T10:37:43.448Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:13.346" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:43.447" grpc.time_ms=0.934 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.147487] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.456Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"23f19a74c5b6d519191a22aa","created_at":"2026-11-13T10:37:43.130Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-9bfe0c326c7b4bd6","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.132Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2485,"job_status":"rescheduled","message":"PipelineProcessWorker JID-23f19a74c5b6d519191a22aa: rescheduled"} gitlab # [ 240.160601] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.462Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"23f19a74c5b6d519191a22aa","created_at":"2026-11-13T10:37:43.130Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-9bfe0c326c7b4bd6","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.132Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-23f19a74c5b6d519191a22aa: done: 0.294736 sec","job_status":"done","queue_duration_s":0.034343,"scheduling_latency_s":0.034358,"gitaly_calls":3,"gitaly_duration_s":0.115165,"redis_calls":69,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.017440999999999998,"redis_read_bytes":226,"redis_write_bytes":18463,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000344,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000853,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.004912,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12309,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.006791,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000158,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000492,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003547,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.079,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.067,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084474,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.268225431,"duration_s":0.294736,"completed_at":"2026-11-13T10:37:43.461Z","load_balancing_strategy":"primary","db_duration_s":0.079753,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 240.204874] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.462Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d884457dcc7bf55bbfe9d089","created_at":"2026-11-13T10:37:43.259Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.261Z","job_size_bytes":23,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d884457dcc7bf55bbfe9d089: start","job_status":"start","queue_duration_s":0.201749,"scheduling_latency_s":0.201764} gitlab # [ 240.218938] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.492Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d884457dcc7bf55bbfe9d089","created_at":"2026-11-13T10:37:43.259Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.261Z","job_size_bytes":23,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d884457dcc7bf55bbfe9d089: done: 0.029101 sec","job_status":"done","queue_duration_s":0.201749,"scheduling_latency_s":0.201764,"redis_calls":7,"redis_duration_s":0.000988,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000713,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01784,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029101,"completed_at":"2026-11-13T10:37:43.491Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008449,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 240.278218] gitaly[906]: time="2026-11-13T10:37:43.495Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R4MQGJKSD3NNV9XRJYEE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.488" pid=906 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 240.278893] gitlab-workhorse[827]: time="2026-11-13T10:37:43Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 240.279434] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:43 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 49 gitlab # [ 240.279903] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.493Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7493ee5ae2ad74caea2455c7","created_at":"2026-11-13T10:37:43.382Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.384Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7493ee5ae2ad74caea2455c7: start","job_status":"start","queue_duration_s":0.109327,"scheduling_latency_s":0.109341} gitlab # [ 240.281972] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.514Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7493ee5ae2ad74caea2455c7","created_at":"2026-11-13T10:37:43.382Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.384Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7493ee5ae2ad74caea2455c7: done: 0.020858 sec","job_status":"done","queue_duration_s":0.109327,"scheduling_latency_s":0.109341,"redis_calls":7,"redis_duration_s":0.001286,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000302,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013741,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020858,"completed_at":"2026-11-13T10:37:43.514Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003403,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.319691] gitaly[906]: time="2026-11-13T10:37:43.495Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R4MQGJKSD3NNV9XRJYEE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.488" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 240.330935] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.515Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"dc2cfb7cc615193f49177bba","created_at":"2026-11-13T10:37:43.394Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.396Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-dc2cfb7cc615193f49177bba: start","job_status":"start","queue_duration_s":0.118776,"scheduling_latency_s":0.118797} gitlab # [ 240.344427] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.533Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"dc2cfb7cc615193f49177bba","created_at":"2026-11-13T10:37:43.394Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.396Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-dc2cfb7cc615193f49177bba: done: 0.017509 sec","job_status":"done","queue_duration_s":0.118776,"scheduling_latency_s":0.118797,"redis_calls":7,"redis_duration_s":0.000877,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000649,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000228,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017509,"completed_at":"2026-11-13T10:37:43.533Z","load_balancing_strategy":"primary","db_duration_s":0.002609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.381068] gitaly[906]: time="2026-11-13T10:37:43.495Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R4MQGJKSD3NNV9XRJYEE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.488" grpc.time_ms=7.661 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 240.393602] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.534Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"abfa64bb5810c7afe015b27d","created_at":"2026-11-13T10:37:43.404Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.405Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-abfa64bb5810c7afe015b27d: start","job_status":"start","queue_duration_s":0.128188,"scheduling_latency_s":0.128202} gitlab # [ 240.435653] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.551Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"abfa64bb5810c7afe015b27d","created_at":"2026-11-13T10:37:43.404Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.405Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-abfa64bb5810c7afe015b27d: done: 0.017783 sec","job_status":"done","queue_duration_s":0.128188,"scheduling_latency_s":0.128202,"redis_calls":10,"redis_duration_s":0.001245,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000626,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000365,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017783,"completed_at":"2026-11-13T10:37:43.551Z","load_balancing_strategy":"primary","db_duration_s":0.002466,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.444141] gitaly[906]: time="2026-11-13T10:37:43.691Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:43.691" grpc.time_ms=0.248 pid=906 span.kind=server system=grpc gitlab # [ 240.452637] gitaly[906]: time="2026-11-13T10:37:43.710Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:43.710" grpc.time_ms=0.189 pid=906 span.kind=server system=grpc gitlab # [ 240.460987] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.552Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"783415bf03093f777d234d45","created_at":"2026-11-13T10:37:43.406Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.407Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-783415bf03093f777d234d45: start","job_status":"start","queue_duration_s":0.1451,"scheduling_latency_s":0.145113} gitlab # [ 240.475913] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.567Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"783415bf03093f777d234d45","created_at":"2026-11-13T10:37:43.406Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.407Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-783415bf03093f777d234d45: done: 0.014449 sec","job_status":"done","queue_duration_s":0.1451,"scheduling_latency_s":0.145113,"redis_calls":7,"redis_duration_s":0.000873,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000654,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000219,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014449,"completed_at":"2026-11-13T10:37:43.567Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002118,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.514923] gitaly[906]: time="2026-11-13T10:37:43.728Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:43.727" grpc.time_ms=0.591 pid=906 span.kind=server system=grpc gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340' gitlab # [ 240.520368] gitaly[906]: time="2026-11-13T10:37:43.737Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST diskcache=46e14412-d94d-473d-a7ac-db29f84eae16 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-13T10:38:13.572" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:43.673" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.527794] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:43 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 60 gitlab # [ 240.529575] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:43 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 44 gitlab # [ 240.531543] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.568Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6ef639ae41ee69b68f97d06c","created_at":"2026-11-13T10:37:43.408Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.409Z","job_size_bytes":19,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6ef639ae41ee69b68f97d06c: start","job_status":"start","queue_duration_s":0.158703,"scheduling_latency_s":0.158717} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340', in 0.05 seconds) gitlab # [ 240.545626] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.582Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"6ef639ae41ee69b68f97d06c","created_at":"2026-11-13T10:37:43.408Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.409Z","job_size_bytes":19,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-6ef639ae41ee69b68f97d06c: done: 0.014032 sec","job_status":"done","queue_duration_s":0.158703,"scheduling_latency_s":0.158717,"redis_calls":8,"redis_duration_s":0.001001,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000775,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000226,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014032,"completed_at":"2026-11-13T10:37:43.582Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002399,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.587103] gitaly[906]: time="2026-11-13T10:37:43.738Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=57 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7129 command.oublock=0 command.real_time_ms=56 command.spawn_token_fork_ms=0 command.system_time_ms=36 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-13T10:38:13.572" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:43.673" grpc.time_ms=64.576 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.591302] gitlab-workhorse[827]: time="2026-11-13T10:37:43Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 240.592744] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:43 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 84 gitlab # [ 240.594472] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.582Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"5108a658efc9effed74f1fc3","created_at":"2026-11-13T10:37:43.411Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.412Z","job_size_bytes":66,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-5108a658efc9effed74f1fc3: start","job_status":"start","queue_duration_s":0.170488,"scheduling_latency_s":0.170527} gitlab # [ 240.637509] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.595Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"5108a658efc9effed74f1fc3","created_at":"2026-11-13T10:37:43.411Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.412Z","job_size_bytes":66,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-5108a658efc9effed74f1fc3: done: 0.012668 sec","job_status":"done","queue_duration_s":0.170488,"scheduling_latency_s":0.170527,"redis_calls":7,"redis_duration_s":0.0008770000000000001,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000635,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012668,"completed_at":"2026-11-13T10:37:43.595Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001441,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.648345] gitaly[906]: time="2026-11-13T10:37:43.783Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-13T10:38:13.682" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:43.782" grpc.time_ms=0.905 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.659781] gitlab-workhorse[827]: time="2026-11-13T10:37:43Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 240.661446] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:43 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 85 gitlab # [ 240.662880] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.596Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d0eac0ecd612d60a2f18c95f","created_at":"2026-11-13T10:37:43.412Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.413Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d0eac0ecd612d60a2f18c95f: start","job_status":"start","queue_duration_s":0.182774,"scheduling_latency_s":0.182787} gitlab # [ 240.677286] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.611Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"d0eac0ecd612d60a2f18c95f","created_at":"2026-11-13T10:37:43.412Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.413Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-d0eac0ecd612d60a2f18c95f: done: 0.015129 sec","job_status":"done","queue_duration_s":0.182774,"scheduling_latency_s":0.182787,"redis_calls":7,"redis_duration_s":0.000899,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000664,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000235,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015129,"completed_at":"2026-11-13T10:37:43.611Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003288,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.732660] nixbot[2602]: {"ts": "2026-11-13T10:37:43.941Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:37968 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 HTTP/1.1\" 200"} gitlab # [ 240.733513] gitaly[906]: time="2026-11-13T10:37:43.807Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7R4Y32JQDN37BG8NJP5W0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=198 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.797" grpc.time_ms=9.811 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 240.734140] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.612Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"38811b121f84fed44373469f","created_at":"2026-11-13T10:37:43.414Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.415Z","job_size_bytes":6,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-38811b121f84fed44373469f: start","job_status":"start","queue_duration_s":0.196384,"scheduling_latency_s":0.196398} gitlab # [ 240.743049] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.620Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"38811b121f84fed44373469f","created_at":"2026-11-13T10:37:43.414Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.415Z","job_size_bytes":6,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-38811b121f84fed44373469f: done: 0.008069 sec","job_status":"done","queue_duration_s":0.196384,"scheduling_latency_s":0.196398,"redis_calls":7,"redis_duration_s":0.000845,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000617,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000228,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008069,"completed_at":"2026-11-13T10:37:43.620Z","load_balancing_strategy":"primary","db_duration_s":0.000262,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.795185] gitaly[906]: time="2026-11-13T10:37:43.808Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R44AGZ045XXG0KPEYVST grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:13.707" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:43.807" grpc.time_ms=0.741 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 240.795813] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.621Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"957018cec6ed6b69b49835da","created_at":"2026-11-13T10:37:43.416Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.417Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-957018cec6ed6b69b49835da: start","job_status":"start","queue_duration_s":0.203662,"scheduling_latency_s":0.203675} gitlab # [ 240.835509] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.634Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"957018cec6ed6b69b49835da","created_at":"2026-11-13T10:37:43.416Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.417Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-957018cec6ed6b69b49835da: done: 0.013021 sec","job_status":"done","queue_duration_s":0.203662,"scheduling_latency_s":0.203675,"redis_calls":7,"redis_duration_s":0.0009010000000000001,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000257,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013021,"completed_at":"2026-11-13T10:37:43.634Z","load_balancing_strategy":"primary","db_duration_s":0.001898,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.840715] gitaly[906]: time="2026-11-13T10:37:43.853Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7R5019Z69HJKNTN225BKQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.845" grpc.time_ms=8.424 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 240.852469] gitaly[906]: time="2026-11-13T10:37:43.893Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R5019Z69HJKNTNDV1850 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.886" pid=906 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 240.861638] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.635Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"0d736c77cea53f29f0667754","created_at":"2026-11-13T10:37:43.456Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.457Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0d736c77cea53f29f0667754: start","job_status":"start","queue_duration_s":0.177569,"scheduling_latency_s":0.177581} gitlab # [ 240.875819] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.820Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"0d736c77cea53f29f0667754","created_at":"2026-11-13T10:37:43.456Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-1276fef516484bd3","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.457Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0d736c77cea53f29f0667754: done: 0.185441 sec","job_status":"done","queue_duration_s":0.177569,"scheduling_latency_s":0.177581,"gitaly_calls":3,"gitaly_duration_s":0.071379,"redis_calls":63,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.012206,"redis_read_bytes":141,"redis_write_bytes":16718,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000306,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000589,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":20,"redis_queues_duration_s":0.002401,"redis_queues_read_bytes":21,"redis_queues_write_bytes":11224,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.004855,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00071,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001821,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001218,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.07595,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.163023234,"duration_s":0.185441,"completed_at":"2026-11-13T10:37:43.820Z","load_balancing_strategy":"primary","db_duration_s":0.024881,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 240.924423] gitaly[906]: time="2026-11-13T10:37:43.893Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R5019Z69HJKNTNDV1850 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.886" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 240.932693] nixbot[2602]: {"ts": "2026-11-13T10:37:44.216Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-bd2d78b9f9a1-5"} gitlab # [ 240.935065] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.821Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c42a5389182b2bef594a5730","created_at":"2026-11-13T10:37:43.668Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.669Z","job_size_bytes":23,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c42a5389182b2bef594a5730: start","job_status":"start","queue_duration_s":0.152115,"scheduling_latency_s":0.152131} gitlab # [ 240.949714] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.851Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c42a5389182b2bef594a5730","created_at":"2026-11-13T10:37:43.668Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.669Z","job_size_bytes":23,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c42a5389182b2bef594a5730: done: 0.030238 sec","job_status":"done","queue_duration_s":0.152115,"scheduling_latency_s":0.152131,"redis_calls":7,"redis_duration_s":0.0011020000000000001,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000347,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030238,"completed_at":"2026-11-13T10:37:43.851Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006483,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 240.991285] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:44 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 233 gitlab # [ 240.991859] gitaly[906]: time="2026-11-13T10:37:43.893Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R5019Z69HJKNTNDV1850 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.886" grpc.time_ms=6.975 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 241.001737] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.853Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"cc7c7dc067604beb25889944","created_at":"2026-11-13T10:37:43.743Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.745Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-cc7c7dc067604beb25889944: start","job_status":"start","queue_duration_s":0.107478,"scheduling_latency_s":0.107496} gitlab # [ 241.016109] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.872Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"cc7c7dc067604beb25889944","created_at":"2026-11-13T10:37:43.743Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.745Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-cc7c7dc067604beb25889944: done: 0.019177 sec","job_status":"done","queue_duration_s":0.107478,"scheduling_latency_s":0.107496,"redis_calls":7,"redis_duration_s":0.0032890000000000003,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00239,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000899,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011422,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019177,"completed_at":"2026-11-13T10:37:43.872Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002489,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.070190] nixbot[2602]: {"ts": "2026-11-13T10:37:44.304Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-bd2d78b9f9a1-5", "argv0": "bwrap"} gitlab # [ 241.070790] gitaly[906]: time="2026-11-13T10:37:43.940Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R51F0GK5Y0FDHTE5QSMT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.911" pid=906 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 241.071278] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.874Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c1f2e0f9bb288cc105ac5f68","created_at":"2026-11-13T10:37:43.751Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.754Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c1f2e0f9bb288cc105ac5f68: start","job_status":"start","queue_duration_s":0.119621,"scheduling_latency_s":0.119635} gitlab # [ 241.078028] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.906Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c1f2e0f9bb288cc105ac5f68","created_at":"2026-11-13T10:37:43.751Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.754Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c1f2e0f9bb288cc105ac5f68: done: 0.031996 sec","job_status":"done","queue_duration_s":0.119621,"scheduling_latency_s":0.119635,"redis_calls":7,"redis_duration_s":0.001999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00156,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031996,"completed_at":"2026-11-13T10:37:43.906Z","load_balancing_strategy":"primary","db_duration_s":0.007426,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.119492] gitaly[906]: time="2026-11-13T10:37:43.941Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R51F0GK5Y0FDHTE5QSMT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.911" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 241.158180] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.907Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"738038ae8451b43494a0c1bb","created_at":"2026-11-13T10:37:43.762Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.764Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-738038ae8451b43494a0c1bb: start","job_status":"start","queue_duration_s":0.143015,"scheduling_latency_s":0.143033} gitlab # [ 241.158839] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.941Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"738038ae8451b43494a0c1bb","created_at":"2026-11-13T10:37:43.762Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.764Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-738038ae8451b43494a0c1bb: done: 0.034047 sec","job_status":"done","queue_duration_s":0.143015,"scheduling_latency_s":0.143033,"redis_calls":10,"redis_duration_s":0.0034920000000000003,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001662,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000428,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001402,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034047,"completed_at":"2026-11-13T10:37:43.941Z","load_balancing_strategy":"primary","db_duration_s":0.009679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.205553] gitaly[906]: time="2026-11-13T10:37:43.941Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R51F0GK5Y0FDHTE5QSMT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-13T10:37:43.911" grpc.time_ms=30.074 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 241.206293] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.942Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"cded94131e27235f35bdcf60","created_at":"2026-11-13T10:37:43.764Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.766Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-cded94131e27235f35bdcf60: start","job_status":"start","queue_duration_s":0.17678,"scheduling_latency_s":0.176797} gitlab # [ 241.206848] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.961Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"cded94131e27235f35bdcf60","created_at":"2026-11-13T10:37:43.764Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.766Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-cded94131e27235f35bdcf60: done: 0.018552 sec","job_status":"done","queue_duration_s":0.17678,"scheduling_latency_s":0.176797,"redis_calls":7,"redis_duration_s":0.001567,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001284,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000283,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018552,"completed_at":"2026-11-13T10:37:43.961Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003961,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.273623] gitaly[906]: time="2026-11-13T10:37:44.020Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:14.018" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:44.019" grpc.time_ms=1.187 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.274421] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.962Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"0a94805f51d715be46c2107b","created_at":"2026-11-13T10:37:43.766Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.768Z","job_size_bytes":19,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-0a94805f51d715be46c2107b: start","job_status":"start","queue_duration_s":0.194007,"scheduling_latency_s":0.19402} gitlab # [ 241.275028] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.991Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"0a94805f51d715be46c2107b","created_at":"2026-11-13T10:37:43.766Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.768Z","job_size_bytes":19,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-0a94805f51d715be46c2107b: done: 0.029235 sec","job_status":"done","queue_duration_s":0.194007,"scheduling_latency_s":0.19402,"redis_calls":8,"redis_duration_s":0.004683,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004369,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000314,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029235,"completed_at":"2026-11-13T10:37:43.991Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007379,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.300777] gitaly[906]: time="2026-11-13T10:37:44.033Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=650 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-13T10:38:14.026" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:44.026" grpc.time_ms=6.088 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.313100] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:43.993Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2e160cca154fd8f25f906567","created_at":"2026-11-13T10:37:43.768Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.769Z","job_size_bytes":66,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2e160cca154fd8f25f906567: start","job_status":"start","queue_duration_s":0.223657,"scheduling_latency_s":0.223676} gitlab # [ 241.328663] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:44.014Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2e160cca154fd8f25f906567","created_at":"2026-11-13T10:37:43.768Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.769Z","job_size_bytes":66,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2e160cca154fd8f25f906567: done: 0.020477 sec","job_status":"done","queue_duration_s":0.223657,"scheduling_latency_s":0.223676,"redis_calls":7,"redis_duration_s":0.001438,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001152,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000286,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020477,"completed_at":"2026-11-13T10:37:44.014Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003042,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.384093] gitaly[906]: time="2026-11-13T10:37:44.048Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=5 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:54.011" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:44.041" grpc.time_ms=7.269 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.384738] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:44.015Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"0c160a84c1ae4db2f06a3bc4","created_at":"2026-11-13T10:37:43.770Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.771Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-0c160a84c1ae4db2f06a3bc4: start","job_status":"start","queue_duration_s":0.243838,"scheduling_latency_s":0.243852} gitlab # [ 241.392430] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:44.031Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"0c160a84c1ae4db2f06a3bc4","created_at":"2026-11-13T10:37:43.770Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.771Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-0c160a84c1ae4db2f06a3bc4: done: 0.016878 sec","job_status":"done","queue_duration_s":0.243838,"scheduling_latency_s":0.243852,"redis_calls":7,"redis_duration_s":0.00105,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016878,"completed_at":"2026-11-13T10:37:44.031Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003931,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.463337] gitaly[906]: time="2026-11-13T10:37:44.071Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-13T10:37:54.041" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:44.071" grpc.time_ms=0.179 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.463944] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:44.032Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a8a750518b12e3eb015b20d9","created_at":"2026-11-13T10:37:43.772Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.773Z","job_size_bytes":6,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a8a750518b12e3eb015b20d9: start","job_status":"start","queue_duration_s":0.259473,"scheduling_latency_s":0.259487} gitlab # [ 241.464573] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:44.044Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a8a750518b12e3eb015b20d9","created_at":"2026-11-13T10:37:43.772Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.773Z","job_size_bytes":6,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a8a750518b12e3eb015b20d9: done: 0.011611 sec","job_status":"done","queue_duration_s":0.259473,"scheduling_latency_s":0.259487,"redis_calls":7,"redis_duration_s":0.001085,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000747,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000338,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011611,"completed_at":"2026-11-13T10:37:44.044Z","load_balancing_strategy":"primary","db_duration_s":0.000259,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.490137] gitaly[906]: time="2026-11-13T10:37:44.080Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=619 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-13T10:37:54.045" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:44.075" grpc.time_ms=4.92 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.529233] gitaly[906]: time="2026-11-13T10:37:44.107Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:44.107" grpc.time_ms=0.236 pid=906 span.kind=server system=grpc gitlab # [ 241.529781] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:44.045Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"73be98826f2d30ff0c8b571c","created_at":"2026-11-13T10:37:43.773Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.774Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-73be98826f2d30ff0c8b571c: start","job_status":"start","queue_duration_s":0.270515,"scheduling_latency_s":0.270528} gitlab # [ 241.530428] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:44.064Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"73be98826f2d30ff0c8b571c","created_at":"2026-11-13T10:37:43.773Z","trace_propagation_headers":{"sentry-trace":"209bf81181204d33bb061e6c4ec013da-fa910a41f81b4e83","baggage":"sentry-trace_id=209bf81181204d33bb061e6c4ec013da,sentry-release=Unknown"},"correlation_id":"01M7A7R44AGZ045XXG0KPEYVST","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:43.774Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-73be98826f2d30ff0c8b571c: done: 0.019165 sec","job_status":"done","queue_duration_s":0.270515,"scheduling_latency_s":0.270528,"redis_calls":7,"redis_duration_s":0.001377,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000368,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019165,"completed_at":"2026-11-13T10:37:44.064Z","load_balancing_strategy":"primary","db_duration_s":0.005435,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.571755] gitaly[906]: time="2026-11-13T10:37:44.124Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:44.124" grpc.time_ms=0.272 pid=906 span.kind=server system=grpc gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340' gitlab # [ 241.572614] gitaly[906]: time="2026-11-13T10:37:44.127Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:37:54.051" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:44.082" operations="[repack_refs]" pid=906 ref=refs/keep-around/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 remote_ip="2001:db8:1::1" revision=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 span.kind=server system=grpc username=root gitlab # [ 241.577217] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:44.144Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"1aa4068d1e3427f585355a10","created_at":"2026-11-13T10:37:44.141Z","trace_propagation_headers":{"sentry-trace":"9af23022b3274e2aa5a48fa91e18f56e-982d498174694e0e","baggage":"sentry-trace_id=9af23022b3274e2aa5a48fa91e18f56e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7R550PH0B7QNBW1RFMKF7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:44.143Z","job_size_bytes":65,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-1aa4068d1e3427f585355a10: start","job_status":"start","queue_duration_s":0.000913,"scheduling_latency_s":0.00095} gitlab # [ 241.618025] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:44.171Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"1aa4068d1e3427f585355a10","created_at":"2026-11-13T10:37:44.141Z","trace_propagation_headers":{"sentry-trace":"9af23022b3274e2aa5a48fa91e18f56e-982d498174694e0e","baggage":"sentry-trace_id=9af23022b3274e2aa5a48fa91e18f56e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7R550PH0B7QNBW1RFMKF7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:44.143Z","job_size_bytes":65,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-1aa4068d1e3427f585355a10: done: 0.027396 sec","job_status":"done","queue_duration_s":0.000913,"scheduling_latency_s":0.00095,"redis_calls":7,"redis_duration_s":0.005134,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004781,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000353,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027396,"completed_at":"2026-11-13T10:37:44.171Z","load_balancing_strategy":"primary","db_duration_s":0.003366,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 241.661842] gitaly[906]: time="2026-11-13T10:37:44.128Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:37:54.051" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:44.082" pid=906 ref=refs/keep-around/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 remote_ip="2001:db8:1::1" repository_info="{false {6 702 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-11-13 10:36:48.080108345 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" revision=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 span.kind=server system=grpc username=root gitlab # [ 241.662919] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:44.173Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"44ee0d182fd44dbe82ec18d3","created_at":"2026-11-13T10:37:44.144Z","trace_propagation_headers":{"sentry-trace":"9af23022b3274e2aa5a48fa91e18f56e-982d498174694e0e","baggage":"sentry-trace_id=9af23022b3274e2aa5a48fa91e18f56e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7R550PH0B7QNBW1RFMKF7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:44.148Z","job_size_bytes":65,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-44ee0d182fd44dbe82ec18d3: start","job_status":"start","queue_duration_s":0.025085,"scheduling_latency_s":0.025099} gitlab # [ 241.663518] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:44.208Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"44ee0d182fd44dbe82ec18d3","created_at":"2026-11-13T10:37:44.144Z","trace_propagation_headers":{"sentry-trace":"9af23022b3274e2aa5a48fa91e18f56e-982d498174694e0e","baggage":"sentry-trace_id=9af23022b3274e2aa5a48fa91e18f56e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7R550PH0B7QNBW1RFMKF7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:44.148Z","job_size_bytes":65,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-44ee0d182fd44dbe82ec18d3: done: 0.035666 sec","job_status":"done","queue_duration_s":0.025085,"scheduling_latency_s":0.025099,"redis_calls":10,"redis_duration_s":0.002318,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001262,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001056,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035666,"completed_at":"2026-11-13T10:37:44.208Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008505,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.693049] gitaly[906]: time="2026-11-13T10:37:44.132Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 diskcache=22a40db2-3a67-4d85-b905-db3f6c3e4f33 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:37:54.051" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:44.082" pid=906 ref=refs/keep-around/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 remote_ip="2001:db8:1::1" revision=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 span.kind=server system=grpc username=root gitlab # [ 241.735215] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:44.210Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"5c785408ca14e7b1a529238e","created_at":"2026-11-13T10:37:44.188Z","trace_propagation_headers":{"sentry-trace":"9af23022b3274e2aa5a48fa91e18f56e-982d498174694e0e","baggage":"sentry-trace_id=9af23022b3274e2aa5a48fa91e18f56e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7R550PH0B7QNBW1RFMKF7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:44.191Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5c785408ca14e7b1a529238e: start","job_status":"start","queue_duration_s":0.019097,"scheduling_latency_s":0.019123} gitlab # [ 241.735873] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:44.296Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"5c785408ca14e7b1a529238e","created_at":"2026-11-13T10:37:44.188Z","trace_propagation_headers":{"sentry-trace":"9af23022b3274e2aa5a48fa91e18f56e-982d498174694e0e","baggage":"sentry-trace_id=9af23022b3274e2aa5a48fa91e18f56e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7R550PH0B7QNBW1RFMKF7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:44.191Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5c785408ca14e7b1a529238e: done: 0.086208 sec","job_status":"done","queue_duration_s":0.019097,"scheduling_latency_s":0.019123,"gitaly_calls":2,"gitaly_duration_s":0.00561,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.005235,"redis_read_bytes":79,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000631,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000678,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000213,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001264,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000156,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000268,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001394,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058935,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.063040973,"duration_s":0.086208,"completed_at":"2026-11-13T10:37:44.296Z","load_balancing_strategy":"primary","db_duration_s":0.018105,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 241.778780] gitaly[906]: time="2026-11-13T10:37:44.133Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=45 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5903 command.oublock=8 command.real_time_ms=45 command.spawn_token_fork_ms=2 command.system_time_ms=33 command.user_time_ms=12 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:37:54.051" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:44.082" grpc.time_ms=50.978 pid=906 ref=refs/keep-around/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 remote_ip="2001:db8:1::1" revision=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 span.kind=server system=grpc username=root gitlab # [ 241.779652] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:44.297Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"7db81e0571b7213827ec0094","created_at":"2026-11-13T10:37:44.251Z","trace_propagation_headers":{"sentry-trace":"9af23022b3274e2aa5a48fa91e18f56e-cf540fad6a8c4fbf","baggage":"sentry-trace_id=9af23022b3274e2aa5a48fa91e18f56e,sentry-release=Unknown"},"correlation_id":"01M7A7R550PH0B7QNBW1RFMKF7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:44.252Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7db81e0571b7213827ec0094: start","job_status":"start","queue_duration_s":0.044887,"scheduling_latency_s":0.044902} gitlab # [ 241.787906] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:44.321Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"7db81e0571b7213827ec0094","created_at":"2026-11-13T10:37:44.251Z","trace_propagation_headers":{"sentry-trace":"9af23022b3274e2aa5a48fa91e18f56e-cf540fad6a8c4fbf","baggage":"sentry-trace_id=9af23022b3274e2aa5a48fa91e18f56e,sentry-release=Unknown"},"correlation_id":"01M7A7R550PH0B7QNBW1RFMKF7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:44.252Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7db81e0571b7213827ec0094: done: 0.023528 sec","job_status":"done","queue_duration_s":0.044887,"scheduling_latency_s":0.044902,"redis_calls":7,"redis_duration_s":0.001104,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000722,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023528,"completed_at":"2026-11-13T10:37:44.321Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003595,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.841529] gitaly[906]: time="2026-11-13T10:37:44.136Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:14.035" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:44.135" grpc.time_ms=0.281 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.842281] gitaly[906]: time="2026-11-13T10:37:44.140Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:37:54.051" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:44.082" pid=906 ref=refs/keep-around/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 remote_ip="2001:db8:1::1" revision=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root gitlab # [ 241.848640] gitaly[906]: time="2026-11-13T10:37:44.141Z" level=info msg="optimized repository" clean_stale_data_time=11.843125ms clean_work_trees_time="68.724µs" commit_graph_time="3.911µs" correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="3.911µs" pid=906 prune_time="3.353µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="4.19µs" storage_name=default system=housekeeping total_time=12.022757ms gitlab # [ 241.853906] gitaly[906]: time="2026-11-13T10:37:44.141Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-13T10:37:54.051" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-13T10:37:44.082" pid=906 ref=refs/keep-around/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 remote_ip="2001:db8:1::1" revision=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 span.kind=server system=grpc username=root gitlab # [ 241.863686] gitaly[906]: time="2026-11-13T10:37:44.267Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-13T10:38:14.164" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:44.265" grpc.time_ms=1.613 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.901102] gitaly[906]: time="2026-11-13T10:37:44.289Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R550PH0B7QNBW1RFMKF7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:14.189" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:44.289" grpc.time_ms=0.417 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.938793] nixbot[2602]: {"ts": "2026-11-13T10:37:45.270Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:37976 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 HTTP/1.1\" 200"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340', in 0.33 seconds) (finished: subtest: push delivers the webhook, in 45.79 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses gitlab # [ 241.975643] nix-daemon[2765]: accepted connection from pid 3621, user nixbot gitlab # [ 241.997191] gitaly[906]: time="2026-11-13T10:37:45.328Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R6EF3ZNPN8D9A22ZY1A4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:15.228" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:45.328" grpc.time_ms=0.408 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 242.017731] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:45 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 53 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses, in 0.07 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses gitlab # [ 243.085529] gitaly[906]: time="2026-11-13T10:37:46.416Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R7GHA1K2FWJE9222GM0C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:16.315" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:46.415" grpc.time_ms=0.783 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.114645] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 60 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses, in 0.10 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses gitlab # [ 244.217287] gitaly[906]: time="2026-11-13T10:37:47.548Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R8JY5SZPHQZX7F7PKECR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:17.446" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:47.547" grpc.time_ms=0.875 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 244.240382] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:47 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 84 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses, in 0.13 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses gitlab # [ 245.352869] gitaly[906]: time="2026-11-13T10:37:48.683Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7R9P8TVQWW947PS2ZGCE6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:18.582" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:48.682" grpc.time_ms=0.993 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.386665] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 101 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses, in 0.14 seconds) gitlab # [ 246.381327] nix-daemon[2765]: accepted connection from pid 3656, user nixbot gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses gitlab # [ 246.393917] nix-daemon[2765]: accepted connection from pid 3621, user nixbot gitlab # [ 246.471136] gitaly[906]: time="2026-11-13T10:37:49.802Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7RASP2QXSQ7XTAJHMVP3E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:19.701" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:49.801" grpc.time_ms=0.634 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.487227] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 67 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses, in 0.11 seconds) gitlab # [ 246.528419] gitaly[906]: time="2026-11-13T10:37:49.860Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:19.759" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:49.859" grpc.time_ms=0.485 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.557238] gitaly[906]: time="2026-11-13T10:37:49.888Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:59.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:49.882" grpc.time_ms=5.833 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.584540] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:49.915Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"7ef471307733bedcc5638a5e","created_at":"2026-11-13T10:37:49.912Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-907bf067aa8d4f3a","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:49.914Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7ef471307733bedcc5638a5e: start","job_status":"start","queue_duration_s":0.001008,"scheduling_latency_s":0.001046} gitlab # [ 246.598144] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:49 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 145 gitlab # [ 246.641769] gitaly[906]: time="2026-11-13T10:37:49.973Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7RAZH57R97VXKGC30V0DD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:19.872" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:49.972" grpc.time_ms=0.684 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.665264] gitaly[906]: time="2026-11-13T10:37:49.997Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7RAZH57R97VXKGC30V0DD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:37:59.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:49.992" grpc.time_ms=4.891 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.683029] gitaly[906]: time="2026-11-13T10:37:50.014Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:50.014" grpc.time_ms=0.295 pid=906 span.kind=server system=grpc gitlab # [ 246.695298] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:50 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 89 gitlab # [ 246.709779] gitaly[906]: time="2026-11-13T10:37:50.041Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:50.041" grpc.time_ms=0.224 pid=906 span.kind=server system=grpc gitlab # [ 246.733660] gitaly[906]: time="2026-11-13T10:37:50.065Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:50.065" grpc.time_ms=0.21 pid=906 span.kind=server system=grpc gitlab # [ 246.749186] gitaly[906]: time="2026-11-13T10:37:50.080Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D diskcache=a37286d9-c5ae-4f80-a563-60f294efb57a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-13T10:38:19.873" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:49.973" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.757914] gitaly[906]: time="2026-11-13T10:37:50.081Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=79 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7180 command.oublock=0 command.real_time_ms=102 command.spawn_token_fork_ms=0 command.system_time_ms=59 command.user_time_ms=20 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-13T10:38:19.873" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:49.973" grpc.time_ms=107.47 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.758581] gitaly[906]: time="2026-11-13T10:37:50.085Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7RB32NEAK9J3CG70203AV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:20.084" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:50.084" grpc.time_ms=0.49 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.798177] gitaly[906]: time="2026-11-13T10:37:50.129Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7RB32NEAK9J3CG70203AV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-13T10:38:00.034" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:50.124" grpc.time_ms=5.107 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.812427] gitaly[906]: time="2026-11-13T10:37:50.144Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-13T10:38:20.043" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:50.143" grpc.time_ms=0.566 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.835032] gitaly[906]: time="2026-11-13T10:37:50.166Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:20.065" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:50.166" grpc.time_ms=0.453 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.847247] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.173Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e5650c753e9bec5e79f0eab0","created_at":"2026-11-13T10:37:50.172Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"7ef471307733bedcc5638a5e","job_size_bytes":3,"pid":2485,"job_status":"deduplicated","message":"PipelineProcessWorker JID-e5650c753e9bec5e79f0eab0: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 246.847689] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.175Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"7ef471307733bedcc5638a5e","created_at":"2026-11-13T10:37:49.912Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-907bf067aa8d4f3a","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:49.914Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2485,"job_status":"rescheduled","message":"PipelineProcessWorker JID-7ef471307733bedcc5638a5e: rescheduled"} gitlab # [ 246.848508] gitlab-workhorse[827]: gitlab 2001:db8:1::1 - - [2026/11/13:10:37:50 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 124 gitlab # [ 246.848962] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.179Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"7ef471307733bedcc5638a5e","created_at":"2026-11-13T10:37:49.912Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-907bf067aa8d4f3a","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:49.914Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7ef471307733bedcc5638a5e: done: 0.263997 sec","job_status":"done","queue_duration_s":0.001008,"scheduling_latency_s":0.001046,"gitaly_calls":3,"gitaly_duration_s":0.113324,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.033675000000000004,"redis_read_bytes":239,"redis_write_bytes":21785,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000609,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001489,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.002437,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14470,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.004235,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000149,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000472,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.023675,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.088467,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.233020119,"duration_s":0.263997,"completed_at":"2026-11-13T10:37:50.179Z","load_balancing_strategy":"primary","db_duration_s":0.029819,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 246.917962] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.181Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"6278b91283d062dcebd8cca9","created_at":"2026-11-13T10:37:49.968Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:49.969Z","job_size_bytes":23,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-6278b91283d062dcebd8cca9: start","job_status":"start","queue_duration_s":0.211692,"scheduling_latency_s":0.21171} gitlab # [ 246.918690] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.205Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"6278b91283d062dcebd8cca9","created_at":"2026-11-13T10:37:49.968Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:49.969Z","job_size_bytes":23,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-6278b91283d062dcebd8cca9: done: 0.0244 sec","job_status":"done","queue_duration_s":0.211692,"scheduling_latency_s":0.21171,"redis_calls":7,"redis_duration_s":0.000871,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000634,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000237,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0244,"completed_at":"2026-11-13T10:37:50.205Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004446,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 246.964699] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.206Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a3f021f6a43b0602ce6712e7","created_at":"2026-11-13T10:37:50.085Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.086Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a3f021f6a43b0602ce6712e7: start","job_status":"start","queue_duration_s":0.119902,"scheduling_latency_s":0.119916} gitlab # [ 246.965430] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.221Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a3f021f6a43b0602ce6712e7","created_at":"2026-11-13T10:37:50.085Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.086Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a3f021f6a43b0602ce6712e7: done: 0.014933 sec","job_status":"done","queue_duration_s":0.119902,"scheduling_latency_s":0.119916,"redis_calls":7,"redis_duration_s":0.000846,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000618,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000228,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014933,"completed_at":"2026-11-13T10:37:50.221Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002303,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 246.985766] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.222Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c277409b745bf18635f29ca4","created_at":"2026-11-13T10:37:50.113Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.117Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c277409b745bf18635f29ca4: start","job_status":"start","queue_duration_s":0.105325,"scheduling_latency_s":0.105339} gitlab # [ 247.032534] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.237Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c277409b745bf18635f29ca4","created_at":"2026-11-13T10:37:50.113Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.117Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c277409b745bf18635f29ca4: done: 0.015258 sec","job_status":"done","queue_duration_s":0.105325,"scheduling_latency_s":0.105339,"redis_calls":7,"redis_duration_s":0.000878,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000651,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000227,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015258,"completed_at":"2026-11-13T10:37:50.237Z","load_balancing_strategy":"primary","db_duration_s":0.002423,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.035872] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.238Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"faf6d706240aa749b102866f","created_at":"2026-11-13T10:37:50.122Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.123Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-faf6d706240aa749b102866f: start","job_status":"start","queue_duration_s":0.114938,"scheduling_latency_s":0.114951} gitlab # [ 247.080700] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.255Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"faf6d706240aa749b102866f","created_at":"2026-11-13T10:37:50.122Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.123Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-faf6d706240aa749b102866f: done: 0.016783 sec","job_status":"done","queue_duration_s":0.114938,"scheduling_latency_s":0.114951,"redis_calls":10,"redis_duration_s":0.001339,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000774,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000338,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000227,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016783,"completed_at":"2026-11-13T10:37:50.255Z","load_balancing_strategy":"primary","db_duration_s":0.002469,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.123878] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.256Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d5365bbb17b8d4863dd41e44","created_at":"2026-11-13T10:37:50.123Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.125Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d5365bbb17b8d4863dd41e44: start","job_status":"start","queue_duration_s":0.130852,"scheduling_latency_s":0.130866} gitlab # [ 247.124660] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.270Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d5365bbb17b8d4863dd41e44","created_at":"2026-11-13T10:37:50.123Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.125Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d5365bbb17b8d4863dd41e44: done: 0.014028 sec","job_status":"done","queue_duration_s":0.130852,"scheduling_latency_s":0.130866,"redis_calls":7,"redis_duration_s":0.000907,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00023,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014028,"completed_at":"2026-11-13T10:37:50.270Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002222,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.174185] gitaly[906]: time="2026-11-13T10:37:50.425Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:50.425" grpc.time_ms=0.286 pid=906 span.kind=server system=grpc gitlab # [ 247.174738] gitaly[906]: time="2026-11-13T10:37:50.443Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:50.442" grpc.time_ms=0.238 pid=906 span.kind=server system=grpc gitlab # [ 247.175598] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.271Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"e654a96f96a93f81f5ab6f71","created_at":"2026-11-13T10:37:50.125Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.127Z","job_size_bytes":19,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-e654a96f96a93f81f5ab6f71: start","job_status":"start","queue_duration_s":0.143737,"scheduling_latency_s":0.14375} gitlab # [ 247.176286] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.284Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"e654a96f96a93f81f5ab6f71","created_at":"2026-11-13T10:37:50.125Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.127Z","job_size_bytes":19,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-e654a96f96a93f81f5ab6f71: done: 0.013261 sec","job_status":"done","queue_duration_s":0.143737,"scheduling_latency_s":0.14375,"redis_calls":8,"redis_duration_s":0.000978,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000758,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00022,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013261,"completed_at":"2026-11-13T10:37:50.284Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.208906] gitaly[906]: time="2026-11-13T10:37:50.460Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-13T10:37:50.460" grpc.time_ms=0.196 pid=906 span.kind=server system=grpc gitlab # [ 247.217444] gitaly[906]: time="2026-11-13T10:37:50.468Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D diskcache=e33b84d3-d47c-42e2-ad21-cf7a8dcc61d0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-13T10:38:20.306" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:50.406" pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.226069] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.285Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"f551c7bf35c71baa42c627a8","created_at":"2026-11-13T10:37:50.127Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.128Z","job_size_bytes":66,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-f551c7bf35c71baa42c627a8: start","job_status":"start","queue_duration_s":0.156288,"scheduling_latency_s":0.156302} gitlab # [ 247.241577] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.305Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"f551c7bf35c71baa42c627a8","created_at":"2026-11-13T10:37:50.127Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.128Z","job_size_bytes":66,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-f551c7bf35c71baa42c627a8: done: 0.020199 sec","job_status":"done","queue_duration_s":0.156288,"scheduling_latency_s":0.156302,"redis_calls":7,"redis_duration_s":0.001094,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000803,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020199,"completed_at":"2026-11-13T10:37:50.305Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001968,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.283959] gitaly[906]: time="2026-11-13T10:37:50.468Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=65 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7528 command.oublock=0 command.real_time_ms=55 command.spawn_token_fork_ms=0 command.system_time_ms=39 command.user_time_ms=26 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-13T10:38:20.306" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-13T10:37:50.406" grpc.time_ms=61.788 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.290628] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.306Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d8f82e1d8d5cf6d13e5c45e6","created_at":"2026-11-13T10:37:50.129Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.131Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d8f82e1d8d5cf6d13e5c45e6: start","job_status":"start","queue_duration_s":0.175905,"scheduling_latency_s":0.175933} gitlab # [ 247.341676] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.327Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d8f82e1d8d5cf6d13e5c45e6","created_at":"2026-11-13T10:37:50.129Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.131Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d8f82e1d8d5cf6d13e5c45e6: done: 0.020399 sec","job_status":"done","queue_duration_s":0.175905,"scheduling_latency_s":0.175933,"redis_calls":7,"redis_duration_s":0.0008990000000000001,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000665,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000234,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014146,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020399,"completed_at":"2026-11-13T10:37:50.327Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00281,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.342747] gitaly[906]: time="2026-11-13T10:37:50.500Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-13T10:38:20.399" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:50.499" grpc.time_ms=0.583 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.381180] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.328Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"97174be189769553ba4e9091","created_at":"2026-11-13T10:37:50.133Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.134Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-97174be189769553ba4e9091: start","job_status":"start","queue_duration_s":0.193643,"scheduling_latency_s":0.193657} gitlab # [ 247.381717] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.342Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"97174be189769553ba4e9091","created_at":"2026-11-13T10:37:50.133Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.134Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-97174be189769553ba4e9091: done: 0.014452 sec","job_status":"done","queue_duration_s":0.193643,"scheduling_latency_s":0.193657,"redis_calls":7,"redis_duration_s":0.000878,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000615,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014452,"completed_at":"2026-11-13T10:37:50.342Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003103,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.440185] gitaly[906]: time="2026-11-13T10:37:50.519Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M7A7RATQXP4Q55CGQE8N735D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-13T10:38:20.418" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-13T10:37:50.519" grpc.time_ms=0.444 pid=906 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.440814] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.343Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3520b9c9b09ca7e6295a79e1","created_at":"2026-11-13T10:37:50.134Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.135Z","job_size_bytes":6,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3520b9c9b09ca7e6295a79e1: start","job_status":"start","queue_duration_s":0.20782,"scheduling_latency_s":0.207832} gitlab # [ 247.441455] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.351Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3520b9c9b09ca7e6295a79e1","created_at":"2026-11-13T10:37:50.134Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.135Z","job_size_bytes":6,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3520b9c9b09ca7e6295a79e1: done: 0.007863 sec","job_status":"done","queue_duration_s":0.20782,"scheduling_latency_s":0.207832,"redis_calls":7,"redis_duration_s":0.000849,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000625,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000224,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005073,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007863,"completed_at":"2026-11-13T10:37:50.351Z","load_balancing_strategy":"primary","db_duration_s":0.000219,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses gitlab # [ 247.464950] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.352Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"9489d96430fe8ba72ab42a39","created_at":"2026-11-13T10:37:50.135Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.136Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-9489d96430fe8ba72ab42a39: start","job_status":"start","queue_duration_s":0.215427,"scheduling_latency_s":0.215439} gitlab # [ 247.512837] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.366Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"9489d96430fe8ba72ab42a39","created_at":"2026-11-13T10:37:50.135Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.136Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-9489d96430fe8ba72ab42a39: done: 0.014089 sec","job_status":"done","queue_duration_s":0.215427,"scheduling_latency_s":0.215439,"redis_calls":7,"redis_duration_s":0.001064,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00072,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000344,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014089,"completed_at":"2026-11-13T10:37:50.366Z","load_balancing_strategy":"primary","db_duration_s":0.001737,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.514000] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.367Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"0bc6673758723281551d3e7b","created_at":"2026-11-13T10:37:50.175Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.176Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0bc6673758723281551d3e7b: start","job_status":"start","queue_duration_s":0.191003,"scheduling_latency_s":0.191022} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340/statuses, in 0.08 seconds) (finished: subtest: eval and build post commit statuses, in 5.64 seconds) (finished: run the VM test script, in 301.41 seconds) test script finished in 301.44s cleanup kill QemuMachine (pid 46) gitlab # [ 247.561531] sidekiq-cluster[2485]: {"severity":"INFO","time":"2026-11-13T10:37:50.527Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"0bc6673758723281551d3e7b","created_at":"2026-11-13T10:37:50.175Z","trace_propagation_headers":{"sentry-trace":"5d43449aaa724972acafe34f0ece58d2-aa6b21e7cb6e473b","baggage":"sentry-trace_id=5d43449aaa724972acafe34f0ece58d2,sentry-release=Unknown"},"correlation_id":"01M7A7RATQXP4Q55CGQE8N735D","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-13T10:37:50.176Z","job_size_bytes":3,"pid":2485,"sidekiq_tid":"m7z5","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0bc6673758723281551d3e7b: done: 0.160102 sec","job_status":"done","queue_duration_s":0.191003,"scheduling_latency_s":0.191022,"gitaly_calls":3,"gitaly_duration_s":0.067367,"redis_calls":65,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.007322,"redis_read_bytes":144,"redis_write_bytes":16876,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000315,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000734,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":20,"redis_queues_duration_s":0.001761,"redis_queues_read_bytes":21,"redis_queues_write_bytes":11225,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.002491,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000149,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000598,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duratqemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14) gitlab # i (finished: cleanup, in 1.39 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [] [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-11-13T10:36:58.644Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-11-13T10:37:43.963336Z', 'started_at': '2026-11-13T10:37:44.210895Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}] [{'id': 5, 'sha': 'bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-13T10:37:44.184Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-13T10:37:44.184Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-13T10:37:44.184Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-13T10:37:44.184Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-13T10:37:44.184Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-11-13T10:37:44.184Z', 'started_at': None, 'finished_at': '2026-11-13T10:37:49.905Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'bd2d78b9f9a131f40b4fd7ffb3f12b87fb5d2340', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-11-13T10:37:50.010Z', 'started_at': None, 'finished_at': '2026-11-13T10:37:50.155Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]