vm-test-run-nixbot-gitlab
checks.x86_64-linux.nixbot-gitlab
· build #452
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab: QEMU running (pid 46)10gitlab # Disk image does not exist, creating the virtualisation disk image...11gitlab # Formatting '/build/vm-state-gitlab/tmp.P67kNXjuSp', fmt=raw size=107374182412(finished: start all VMs, in 52.09 seconds)13gitlab # mke2fs 1.47.4 (6-Mar-2025)14gitlab: waiting for unit setup-gitlab.service15gitlab # Discarding device blocks: 0/262144 done16gitlab: waiting for the VM to finish booting17gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes18gitlab # Filesystem UUID: 6e91375e-6b75-4e91-929f-1282cc2cdd2f19gitlab # Superblock backups stored on blocks:20gitlab # 32768, 98304, 163840, 22937621gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/1fhpq8fg97y6xiqyn6b5nfdx60qaisig-nixos-system-gitlab-test/init regInfo=/nix/store/4ck8rn3m8md7bhkixnl9dm2vxj7ff3qk-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000000] kvm-clock: using sched offset of 491749823 cycles65gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000005] tsc: Detected 2400.012 MHz processor67gitlab # [ 0.000804] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000831] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.000872] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.002730] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.002740] Using GB pages for direct mapping73gitlab # [ 0.002855] RAMDISK: [mem 0xbe31c000-0xbffcffff]74gitlab # [ 0.002863] ACPI: Early table checksum verification disabled75gitlab # [ 0.002866] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.002870] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.002873] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002881] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002883] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.002884] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002886] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002887] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002889] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.002890] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.002890] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.002891] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.002891] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.002892] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003121] No NUMA configuration found90gitlab # [ 0.003122] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003124] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff]92gitlab # [ 0.005603] Zone ranges:93gitlab # [ 0.005604] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.005606] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.005607] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.005608] Device empty97gitlab # [ 0.005609] Movable zone start for each node98gitlab # [ 0.005610] Early memory node ranges99gitlab # [ 0.005610] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.005611] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.005612] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.005613] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.005633] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.005926] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.123076] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.123523] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.123539] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.123577] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.123580] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.123581] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.123583] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.123584] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.123585] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.123587] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.123589] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.123594] TSC deadline timer available117gitlab # [ 0.123599] CPU topo: Max. logical packages: 1118gitlab # [ 0.123600] CPU topo: Max. logical dies: 1119gitlab # [ 0.123600] CPU topo: Max. dies per package: 1120gitlab # [ 0.123604] CPU topo: Max. threads per core: 1121gitlab # [ 0.123604] CPU topo: Num. cores per package: 4122gitlab # [ 0.123605] CPU topo: Num. threads per package: 4123gitlab # [ 0.123605] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.123631] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.123650] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.123653] kvm-guest: setup PV sched yield127gitlab # [ 0.123669] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.123671] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.123672] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.123674] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.123676] Booting paravirtualized kernel on KVM132gitlab # [ 0.123680] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.128176] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.130546] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.130601] kvm-guest: PV spinlocks enabled136gitlab # [ 0.130603] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.130607] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/1fhpq8fg97y6xiqyn6b5nfdx60qaisig-nixos-system-gitlab-test/init regInfo=/nix/store/4ck8rn3m8md7bhkixnl9dm2vxj7ff3qk-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.130707] Unknown kernel command line parameters "regInfo=/nix/store/4ck8rn3m8md7bhkixnl9dm2vxj7ff3qk-closure-info/registration", will be passed to user space.139gitlab # [ 0.130720] random: crng init done140gitlab # [ 0.130721] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.140358] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.145371] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.145452] software IO TLB: area num 4.144gitlab # [ 0.226641] Fallback order for Node 0: 0145gitlab # [ 0.226650] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.226652] Policy zone: Normal147gitlab # [ 0.229289] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.239770] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.253455] allocated 12582912 bytes of page_ext150gitlab # [ 0.263599] ftrace: allocating 48717 entries in 192 pages151gitlab # [ 0.263602] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.264512] Dynamic Preempt: lazy153gitlab # [ 0.264686] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.264687] rcu: RCU event tracing is enabled.155gitlab # [ 0.264688] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.264689] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.264690] Rude variant of Tasks RCU enabled.158gitlab # [ 0.264690] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.264691] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.264691] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.264708] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.264710] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.264711] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.269122] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.269410] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.269419] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns167gitlab # [ 0.269540] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)168gitlab # [ 0.273121] Console: colour VGA+ 80x25169gitlab # [ 0.273124] printk: legacy console [tty0] enabled170gitlab # [ 0.304098] printk: legacy console [ttyS0] enabled171gitlab # [ 0.412269] ACPI: Core revision 20250807172gitlab # [ 0.413245] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns173gitlab # [ 0.414864] APIC: Switch to symmetric I/O mode setup174gitlab # [ 0.415932] x2apic enabled175gitlab # [ 0.416683] APIC: Switched APIC routing to: physical x2apic176gitlab # [ 0.417655] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()177gitlab # [ 0.418964] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()178gitlab # [ 0.420440] kvm-guest: setup PV IPIs179gitlab # [ 0.422123] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1180gitlab # [ 0.423178] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns181gitlab # [ 0.424980] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)182gitlab # [ 0.426066] x86/cpu: User Mode Instruction Prevention (UMIP) activated183gitlab # [ 0.427098] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127184gitlab # [ 0.428977] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0185gitlab # [ 0.429981] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto186gitlab # [ 0.430977] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl187gitlab # [ 0.431977] Transient Scheduler Attacks: Vulnerable: No microcode188gitlab # [ 0.433762] Spectre V2 : Mitigation: Enhanced / Automatic IBRS189gitlab # [ 0.434721] Speculative Return Stack Overflow: Mitigation: Safe RET190gitlab # [ 0.435780] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization191gitlab # [ 0.436985] Spectre V2 : Enabling IBPB for BPF192gitlab # [ 0.437978] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier193gitlab # [ 0.438978] active return thunk: srso_alias_return_thunk194gitlab # [ 0.439998] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'195gitlab # [ 0.441975] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'196gitlab # [ 0.442977] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'197gitlab # [ 0.443977] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'198gitlab # [ 0.444977] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'199gitlab # [ 0.445977] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'200gitlab # [ 0.446977] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'201gitlab # [ 0.447977] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256202gitlab # [ 0.449775] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64203gitlab # [ 0.450977] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512204gitlab # [ 0.451976] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024205gitlab # [ 0.452977] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8206gitlab # [ 0.453977] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.207gitlab # [ 0.484459] Freeing SMP alternatives memory: 44K208gitlab # [ 0.484981] pid_max: default: 32768 minimum: 301209gitlab # [ 0.486083] LSM: initializing lsm=capability,landlock,yama,bpf,ima210gitlab # [ 0.487085] landlock: Up and running.211gitlab # [ 0.487977] Yama: becoming mindful.212gitlab # [ 0.489056] LSM support for eBPF active213gitlab # [ 0.489954] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)214gitlab # [ 0.491126] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)215gitlab # [ 0.494144] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)216gitlab # [ 0.495578] Performance Events: Fam17h+ core perfctr, AMD PMU driver.217gitlab # [ 0.495987] ... version: 2218gitlab # [ 0.496765] ... bit width: 48219gitlab # [ 0.496979] ... generic counters: 6220gitlab # [ 0.497727] ... generic bitmap: 000000000000003f221gitlab # [ 0.497979] ... fixed-purpose counters: 0222gitlab # [ 0.498760] ... fixed-purpose bitmap: 0000000000000000223gitlab # [ 0.498979] ... value mask: 0000ffffffffffff224gitlab # [ 0.499948] ... max period: 00007fffffffffff225gitlab # [ 0.500751] ... global_ctrl mask: 000000000000003f226gitlab # [ 0.501086] signal: max sigframe size: 3376227gitlab # [ 0.501941] rcu: Hierarchical SRCU implementation.228gitlab # [ 0.502642] rcu: Max phase no-delay instances is 400.229gitlab # [ 0.503169] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level230gitlab # [ 0.508556] smp: Bringing up secondary CPUs ...231gitlab # [ 0.509379] smpboot: x86: Booting SMP configuration:232gitlab # [ 0.509983] .... node #0, CPUs: #1 #2 #3233gitlab # [ 0.514125] smp: Brought up 1 node, 4 CPUs234gitlab # [ 0.515728] smpboot: Total of 4 processors activated (19200.09 BogoMIPS)235gitlab # [ 0.516310] Memory: 6011636K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260108K reserved, 0K cma-reserved)236gitlab # [ 0.517980] devtmpfs: initialized237gitlab # [ 0.519025] x86/mm: Memory block size: 128MB238gitlab # [ 0.523013] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)239gitlab # [ 0.524062] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).240gitlab # [ 0.525076] pinctrl core: initialized pinctrl subsystem241gitlab # [ 0.526320] PM: RTC time: 10:58:22, date: 2026-10-10242gitlab # [ 0.530203] NET: Registered PF_NETLINK/PF_ROUTE protocol family243gitlab # [ 0.532374] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations244gitlab # [ 0.533050] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations245gitlab # [ 0.535181] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations246gitlab # [ 0.535992] audit: initializing netlink subsys (disabled)247gitlab # [ 0.537012] audit: type=2000 audit(1791629902.133:1): state=initialized audit_enabled=0 res=1248gitlab # [ 0.537257] thermal_sys: Registered thermal governor 'fair_share'249gitlab # [ 0.538980] thermal_sys: Registered thermal governor 'bang_bang'250gitlab # [ 0.539979] thermal_sys: Registered thermal governor 'step_wise'251gitlab # [ 0.540979] thermal_sys: Registered thermal governor 'user_space'252gitlab # [ 0.541970] thermal_sys: Registered thermal governor 'power_allocator'253gitlab # [ 0.542779] cpuidle: using governor menu254gitlab # [ 0.545251] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5255gitlab # [ 0.546240] PCI: Using configuration type 1 for base access256gitlab # [ 0.546980] PCI: Using configuration type 1 for extended access257gitlab # [ 0.548156] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.258gitlab # [ 0.550022] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages259gitlab # [ 0.550980] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page260gitlab # [ 0.551980] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages261gitlab # [ 0.552979] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page262gitlab # [ 0.558225] ACPI: Added _OSI(Module Device)263gitlab # [ 0.559988] ACPI: Added _OSI(Processor Device)264gitlab # [ 0.561988] ACPI: Added _OSI(Processor Aggregator Device)265gitlab # [ 0.568920] ACPI: 1 ACPI AML tables successfully acquired and loaded266gitlab # [ 0.575974] ACPI: Interpreter enabled267gitlab # [ 0.578075] ACPI: PM: (supports S0 S3 S4 S5)268gitlab # [ 0.579988] ACPI: Using IOAPIC for interrupt routing269gitlab # [ 0.582169] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug270gitlab # [ 0.585986] PCI: Using E820 reservations for host bridge windows271gitlab # [ 0.588467] ACPI: Enabled 2 GPEs in block 00 to 0F272gitlab # [ 0.600471] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])273gitlab # [ 0.602995] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]274gitlab # [ 0.606900] acpiphp: Slot [3] registered275gitlab # [ 0.608026] acpiphp: Slot [4] registered276gitlab # [ 0.609038] acpiphp: Slot [5] registered277gitlab # [ 0.611033] acpiphp: Slot [6] registered278gitlab # [ 0.612033] acpiphp: Slot [7] registered279gitlab # [ 0.614063] acpiphp: Slot [8] registered280gitlab # [ 0.615032] acpiphp: Slot [9] registered281gitlab # [ 0.616032] acpiphp: Slot [10] registered282gitlab # [ 0.618009] acpiphp: Slot [11] registered283gitlab # [ 0.618790] acpiphp: Slot [12] registered284gitlab # [ 0.619006] acpiphp: Slot [13] registered285gitlab # [ 0.619746] acpiphp: Slot [14] registered286gitlab # [ 0.621001] acpiphp: Slot [15] registered287gitlab # [ 0.621767] acpiphp: Slot [16] registered288gitlab # [ 0.621999] acpiphp: Slot [17] registered289gitlab # [ 0.622729] acpiphp: Slot [18] registered290gitlab # [ 0.624009] acpiphp: Slot [19] registered291gitlab # [ 0.624767] acpiphp: Slot [20] registered292gitlab # [ 0.624999] acpiphp: Slot [21] registered293gitlab # [ 0.625746] acpiphp: Slot [22] registered294gitlab # [ 0.627010] acpiphp: Slot [23] registered295gitlab # [ 0.627762] acpiphp: Slot [24] registered296gitlab # [ 0.627999] acpiphp: Slot [25] registered297gitlab # [ 0.628730] acpiphp: Slot [26] registered298gitlab # [ 0.630032] acpiphp: Slot [27] registered299gitlab # [ 0.630787] acpiphp: Slot [28] registered300gitlab # [ 0.630999] acpiphp: Slot [29] registered301gitlab # [ 0.631724] acpiphp: Slot [30] registered302gitlab # [ 0.633015] acpiphp: Slot [31] registered303gitlab # [ 0.633787] PCI host bridge to bus 0000:00304gitlab # [ 0.633985] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]305gitlab # [ 0.634980] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]306gitlab # [ 0.635985] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]307gitlab # [ 0.637980] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]308gitlab # [ 0.638980] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]309gitlab # [ 0.640029] pci_bus 0000:00: root bus resource [bus 00-ff]310gitlab # [ 0.641494] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint311gitlab # [ 0.643588] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint312gitlab # [ 0.648122] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint313gitlab # [ 0.655071] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]314gitlab # [ 0.657054] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk315gitlab # [ 0.659985] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk316gitlab # [ 0.661985] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk317gitlab # [ 0.664985] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk318gitlab # [ 0.668462] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint319gitlab # [ 0.675189] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]320gitlab # [ 0.678201] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint321gitlab # [ 0.681551] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI322gitlab # [ 0.683995] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB323gitlab # [ 0.687409] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint324gitlab # [ 0.705231] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]325gitlab # [ 0.708031] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]326gitlab # [ 0.711061] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]327gitlab # [ 0.715832] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]328gitlab # [ 0.721141] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint329gitlab # [ 0.732021] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]330gitlab # [ 0.735001] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]331gitlab # [ 0.737050] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]332gitlab # [ 0.766000] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]333gitlab # [ 0.771171] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint334gitlab # [ 0.778018] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]335gitlab # [ 0.778987] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]336gitlab # [ 0.780002] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]337gitlab # [ 0.783648] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint338gitlab # [ 0.788013] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]339gitlab # [ 0.788937] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]340gitlab # [ 0.790795] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]341gitlab # [ 0.793197] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint342gitlab # [ 0.797012] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]343gitlab # [ 0.798987] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]344gitlab # [ 0.800002] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]345gitlab # [ 0.802242] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint346gitlab # [ 0.806015] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]347gitlab # [ 0.806950] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]348gitlab # [ 0.807792] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]349gitlab # [ 0.811264] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint350gitlab # [ 0.814935] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]351gitlab # [ 0.815704] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]352gitlab # [ 0.817028] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]353gitlab # [ 0.819891] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint354gitlab # [ 0.823016] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]355gitlab # [ 0.823941] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]356gitlab # [ 0.824803] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]357gitlab # [ 0.825968] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]358gitlab # [ 0.828086] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint359gitlab # [ 0.831582] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]360gitlab # [ 0.833003] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]361gitlab # [ 0.835239] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint362gitlab # [ 0.839025] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]363gitlab # [ 0.839953] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]364gitlab # [ 0.841053] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]365gitlab # [ 0.844552] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint366gitlab # [ 0.852028] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]367gitlab # [ 0.855001] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]368gitlab # [ 0.857040] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]369gitlab # [ 0.869575] ACPI: PCI: Interrupt link LNKA configured for IRQ 10370gitlab # [ 0.872381] ACPI: PCI: Interrupt link LNKB configured for IRQ 10371gitlab # [ 0.875328] ACPI: PCI: Interrupt link LNKC configured for IRQ 11372gitlab # [ 0.878317] ACPI: PCI: Interrupt link LNKD configured for IRQ 11373gitlab # [ 0.881172] ACPI: PCI: Interrupt link LNKS configured for IRQ 9374gitlab # [ 0.887057] iommu: Default domain type: Translated375gitlab # [ 0.887986] iommu: DMA domain TLB invalidation policy: lazy mode376gitlab # [ 0.891530] ACPI: bus type USB registered377gitlab # [ 0.893184] usbcore: registered new interface driver usbfs378gitlab # [ 0.893998] usbcore: registered new interface driver hub379gitlab # [ 0.894986] usbcore: registered new device driver usb380gitlab # [ 0.897556] NetLabel: Initializing381gitlab # [ 0.897981] NetLabel: domain hash size = 128382gitlab # [ 0.898765] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO383gitlab # [ 0.900014] NetLabel: unlabeled traffic allowed by default384gitlab # [ 0.900989] PCI: Using ACPI for IRQ routing385gitlab # [ 0.902249] pci 0000:00:02.0: vgaarb: setting as boot VGA device386gitlab # [ 0.902976] pci 0000:00:02.0: vgaarb: bridge control possible387gitlab # [ 0.902976] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none388gitlab # [ 0.905998] vgaarb: loaded389gitlab # [ 0.907018] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0390gitlab # [ 0.908979] hpet0: 3 comparators, 64-bit 100.000000 MHz counter391gitlab # [ 0.913181] clocksource: Switched to clocksource kvm-clock392gitlab # [ 0.915111] VFS: Disk quotas dquot_6.6.0393gitlab # [ 0.915877] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)394gitlab # [ 0.917472] pnp: PnP ACPI init395gitlab # [ 0.918709] pnp: PnP ACPI: found 6 devices396gitlab # [ 0.928446] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns397gitlab # [ 0.930034] clocksource: Switched to clocksource acpi_pm398gitlab # [ 0.931132] NET: Registered PF_INET protocol family399gitlab # [ 0.933782] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)400gitlab # [ 0.952344] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)401gitlab # [ 0.953966] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)402gitlab # [ 0.955386] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)403gitlab # [ 0.959238] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)404gitlab # [ 0.960649] TCP: Hash tables configured (established 65536 bind 65536)405gitlab # [ 0.961862] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)406gitlab # [ 0.963238] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)407gitlab # [ 0.964747] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)408gitlab # [ 0.966128] NET: Registered PF_UNIX/PF_LOCAL protocol family409gitlab # [ 0.967152] NET: Registered PF_XDP protocol family410gitlab # [ 0.968052] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]411gitlab # [ 0.969119] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]412gitlab # [ 0.970190] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]413gitlab # [ 0.971334] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]414gitlab # [ 0.972537] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]415gitlab # [ 0.973940] pci 0000:00:01.0: PIIX3: Enabling Passive Release416gitlab # [ 0.974961] pci 0000:00:00.0: Limiting direct PCI/PCI transfers417gitlab # [ 0.977363] ACPI: \_SB_.LNKD: Enabled at IRQ 11418gitlab # [ 0.979936] PCI: CLS 0 bytes, default 64419gitlab # [ 0.980765] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)420gitlab # [ 0.981125] Trying to unpack rootfs image as initramfs...421gitlab # [ 0.981873] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB)422gitlab # [ 0.982002] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns423gitlab # [ 1.005043] Initialise system trusted keyrings424gitlab # [ 1.006346] workingset: timestamp_bits=40 max_order=21 bucket_order=0425gitlab # [ 1.017944] Key type asymmetric registered426gitlab # [ 1.018743] Asymmetric key parser 'x509' registered427gitlab # [ 1.019725] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)428gitlab # [ 1.021417] io scheduler mq-deadline registered429gitlab # [ 1.022228] io scheduler kyber registered430gitlab # [ 1.026222] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled431gitlab # [ 1.027634] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A432gitlab # [ 1.030091] Linux agpgart interface v0.103433gitlab # [ 1.030917] ACPI: bus type drm_connector registered434gitlab # [ 1.033429] usbcore: registered new interface driver usbserial_generic435gitlab # [ 1.034593] usbserial: USB Serial support registered for generic436gitlab # [ 1.035626] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.437gitlab # [ 1.035626] Please enable it if your BIOS has the CPPC option.438gitlab # [ 1.037953] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled439gitlab # [ 1.039389] drop_monitor: Initializing network drop monitor service440gitlab # [ 1.040652] NET: Registered PF_INET6 protocol family441gitlab # [ 1.042325] Segment Routing with IPv6442gitlab # [ 1.043041] In-situ OAM (IOAM) with IPv6443gitlab # [ 1.045084] IPI shorthand broadcast: enabled444gitlab # [ 1.049474] sched_clock: Marking stable (902010537, 146556792)->(1074052470, -25485141)445gitlab # [ 1.051313] registered taskstats version 1446gitlab # [ 1.052375] Loading compiled-in X.509 certificates447gitlab # [ 1.061983] Demotion targets for Node 0: null448gitlab # [ 1.063156] Key type .fscrypt registered449gitlab # [ 1.063903] Key type fscrypt-provisioning registered450gitlab # [ 1.064916] ima: No TPM chip found, activating TPM-bypass!451gitlab # [ 1.065882] ima: Allocated hash algorithm: sha1452gitlab # [ 1.066746] ima: No architecture policies found453gitlab # [ 1.067813] PM: Magic number: 14:278:982454gitlab # [ 1.069536] RAS: Correctable Errors collector initialized.455gitlab # [ 1.074019] clk: Disabling unused clocks456gitlab # [ 1.074766] PM: genpd: Disabling unused power domains457gitlab # [ 1.276956] Freeing initrd memory: 29392K458gitlab # [ 1.280723] Freeing unused decrypted memory: 2028K459gitlab # [ 1.283710] Freeing unused kernel image (initmem) memory: 3640K460gitlab # [ 1.284864] Write protecting the kernel read-only data: 32768k461gitlab # [ 1.287050] Freeing unused kernel image (text/rodata gap) memory: 1220K462gitlab # [ 1.288792] Freeing unused kernel image (rodata/data gap) memory: 760K463gitlab # [ 1.340253] x86/mm: Checked W+X mappings: passed, no W+X pages found.464gitlab # [ 1.341494] Run /init as init process465gitlab # [ 1.354725] systemd[1]: Inserted module 'autofs4'466gitlab # [ 1.378072] fuse: init (API version 7.45)467gitlab # [ 1.386173] ACPI: \_SB_.LNKC: Enabled at IRQ 10468gitlab # [ 1.393222] ACPI: \_SB_.LNKA: Enabled at IRQ 10469gitlab # [ 1.396708] ACPI: \_SB_.LNKB: Enabled at IRQ 11470gitlab # [ 1.434067] systemd[1]: Successfully made /usr/ read-only.471gitlab # [ 1.770028] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)472gitlab # [ 1.799412] systemd[1]: Detected virtualization kvm.473gitlab # [ 1.801640] systemd[1]: Detected architecture x86-64.474gitlab # [ 1.803807] systemd[1]: Running in initrd.475gitlab # [ 1.806247] systemd[1]: Initializing machine ID from random generator.476gitlab # [ 1.808878] systemd[1]: Hostname set to <gitlab>.477gitlab # [ 1.933101] systemd[1]: bpf-restrict-fs: LSM BPF program attached478gitlab # [ 1.995354] systemd[1]: Queued start job for default target Initrd Default Target.479gitlab # [ 2.006187] systemd[1]: Created slice Slice /system/modprobe.480gitlab # [ 2.007368] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.481gitlab # [ 2.008857] systemd[1]: Expecting device /dev/disk/by-label/nix-store...482gitlab # [ 2.010030] systemd[1]: Expecting device /dev/disk/by-label/nixos...483gitlab # [ 2.011173] systemd[1]: Reached target Path Units.484gitlab # [ 2.012082] systemd[1]: Reached target Slice Units.485gitlab # [ 2.013003] systemd[1]: Reached target Swaps.486gitlab # [ 2.013847] systemd[1]: Reached target Timer Units.487gitlab # [ 2.014848] systemd[1]: Listening on D-Bus System Message Bus Socket.488gitlab # [ 2.016091] systemd[1]: Listening on Journal Socket (/dev/log).489gitlab # [ 2.017315] systemd[1]: Listening on Journal Sockets.490gitlab # [ 2.018333] systemd[1]: Listening on udev Control Socket.491gitlab # [ 2.019390] systemd[1]: Listening on udev Kernel Socket.492gitlab # [ 2.020352] systemd[1]: Reached target Socket Units.493gitlab # [ 2.022278] systemd[1]: Starting Create List of Static Device Nodes...494gitlab # [ 2.024491] systemd[1]: Starting Load Kernel Module 9pnet_virtio...495gitlab # [ 2.026719] systemd[1]: Starting Load Kernel Module configfs...496gitlab # [ 2.029463] systemd[1]: Starting Journal Service...497gitlab # [ 2.033643] systemd[1]: Starting Load Kernel Modules...498gitlab # [ 2.036003] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os499gitlab # [ 2.041651] systemd[1]: Starting Coldplug All udev Devices...500gitlab # [ 2.046684] systemd[1]: Finished Create List of Static Device Nodes.501gitlab # [ 2.048526] systemd[1]: modprobe@configfs.service: Deactivated successfully.502gitlab # [ 2.052983] systemd[1]: Finished Load Kernel Module configfs.503gitlab # [ 2.058047] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config504gitlab # [ 2.060124] systemd-journald[92]: Collecting audit messages is disabled.505gitlab # [ 2.067641] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...506gitlab # [ 2.086274] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.507gitlab # [ 2.091779] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev508gitlab # [ 2.093908] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.509gitlab # [ 2.095106] netfs: FS-Cache loaded510gitlab # [ 2.096149] systemd[1]: Finished Load Kernel Modules.511gitlab # [ 2.098363] systemd[1]: Starting Apply Kernel Variables...512gitlab # [ 2.101934] 9pnet: Installing 9P2000 support513gitlab # [ 2.102717] systemd[1]: Starting Create Static Device Nodes in /dev...514gitlab # [ 2.107375] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.515gitlab # [ 2.112878] systemd[1]: Finished Load Kernel Module 9pnet_virtio.516gitlab # [ 1.966483] systemd-modules-load[93]: Using 4 probe threads517gitlab # [ 2.114678] systemd[1]: Started Journal Service.518gitlab # [ 1.968883] systemd-modules-load[93]: Inserted module 'virtio_balloon'519gitlab # [ 1.970198] systemd-modules-load[93]: Inserted module 'virtio_gpu'520gitlab # [ 1.971989] systemd-modules-load[93]: Inserted module 'dm_mod'521gitlab # [ 1.974291] systemd[1]: Finished Apply Kernel Variables.522gitlab # [ 1.975424] systemd[1]: Finished Create Static Device Nodes in /dev.523gitlab # [ 1.977654] systemd[1]: Reached target Preparation for Local File Systems.524gitlab # [ 1.978640] systemd[1]: Reached target Local File Systems.525gitlab # [ 1.979472] systemd[1]: Starting Create System Files and Directories...526gitlab # [ 1.980431] systemd[1]: Starting Rule-based Manager for Device Events and Files...527gitlab # [ 1.992586] systemd[1]: Finished Create System Files and Directories.528gitlab # [ 2.003535] systemd-udevd[112]: Using default interface naming scheme 'v261'.529gitlab # [ 2.013914] systemd[1]: Started Rule-based Manager for Device Events and Files.530gitlab # [ 2.022975] systemd[1]: Finished Coldplug All udev Devices.531gitlab # [ 2.023769] systemd[1]: Reached target System Initialization.532gitlab # [ 2.024608] systemd[1]: Reached target Basic System.533gitlab # [ 2.276054] virtio_blk virtio4: 4/0/0 default/read/poll queues534gitlab # [ 2.279483] SCSI subsystem initialized535gitlab # [ 2.283151] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12536gitlab # [ 2.283405] uhci_hcd 0000:00:01.2: UHCI Host Controller537gitlab # [ 2.285526] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1538gitlab # [ 2.286996] serio: i8042 KBD port at 0x60,0x64 irq 1539gitlab # [ 2.287763] serio: i8042 AUX port at 0x60,0x64 irq 12540gitlab # [ 2.288596] uhci_hcd 0000:00:01.2: detected 2 ports541gitlab # [ 2.289458] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140542gitlab # [ 2.290967] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18543gitlab # [ 2.292214] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1544gitlab # [ 2.293378] usb usb1: Product: UHCI Host Controller545gitlab # [ 2.294370] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd546gitlab # [ 2.295192] usb usb1: SerialNumber: 0000:00:01.2547gitlab # [ 2.298967] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)548gitlab # [ 2.302442] hub 1-0:1.0: USB hub found549gitlab # [ 2.303163] hub 1-0:1.0: 2 ports detected550gitlab # [ 2.303260] virtio_blk virtio5: 4/0/0 default/read/poll queues551gitlab # [ 2.307377] virtio_blk virtio5: [vdb] 11953192 512-byte logical blocks (6.12 GB/5.70 GiB)552gitlab # [ 2.323738] scsi host0: ata_piix553gitlab # [ 2.332467] scsi host1: ata_piix554gitlab # [ 2.358621] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0555gitlab # [ 2.359832] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0556gitlab # [ 2.222403] systemd[1]: Starting Virtual Console Setup...557gitlab # [ 2.228090] systemd[1]: Found device /dev/disk/by-label/nixos.558gitlab # [ 2.229161] systemd[1]: Reached target Initrd Root Device.559gitlab # [ 2.230644] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...560gitlab # [ 2.235302] systemd[1]: Found device /dev/disk/by-label/nix-store.561gitlab # [ 2.246373] systemd-vconsole-setup[150]: Configuration of first virtual console was skipped, ignoring remaining ones.562gitlab # [ 2.248174] systemd[1]: Finished Virtual Console Setup.563gitlab # [ 2.251949] systemd-fsck[155]: nixos: clean, 12/65536 files, 13019/262144 blocks564gitlab # [ 2.255460] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.565gitlab # [ 2.520106] usb 1-1: new full-speed USB device number 2 using uhci_hcd566gitlab # [ 2.520303] ata2: found unknown device (class 0)567gitlab # [ 2.525537] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100568gitlab # [ 2.529881] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5569gitlab # [ 2.560897] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0570gitlab # [ 2.418230] (udev-worker)[125]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.571gitlab # [ 2.423612] (udev-worker)[125]: Network interface NamePolicy= disabled on kernel command line.572gitlab # [ 2.427199] (udev-worker)[140]: Network interface NamePolicy= disabled on kernel command line.573gitlab # [ 2.598882] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray574gitlab # [ 2.628285] cdrom: Uniform CD-ROM driver Revision: 3.20575gitlab # [ 2.687876] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00576gitlab # [ 2.690926] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10577gitlab # [ 2.693633] usb 1-1: Product: QEMU USB Tablet578gitlab # [ 2.695220] usb 1-1: Manufacturer: QEMU579gitlab # [ 2.696814] usb 1-1: SerialNumber: 28754-0000:00:01.2-1580gitlab # [ 2.728736] hid: raw HID events driver (C) Jiri Kosina581gitlab # [ 2.743105] usbcore: registered new interface driver usbhid582gitlab # [ 2.745768] usbhid: USB HID core driver583gitlab # [ 2.755463] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2584gitlab # [ 2.758996] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0585gitlab # [ 2.900371] systemd[1]: Mounting /sysroot...586gitlab # [ 3.232408] EXT4-fs (vda): mounted filesystem 6e91375e-6b75-4e91-929f-1282cc2cdd2f r/w with ordered data mode. Quota mode: none.587gitlab # [ 3.093667] systemd[1]: Mounted /sysroot.588gitlab # [ 3.095612] systemd[1]: Reached target Initrd Root File System.589gitlab # [ 3.099587] systemd[1]: Mounting /sysroot/nix/.ro-store...590gitlab # [ 3.101840] systemd[1]: Mounting /sysroot/nix/.rw-store...591gitlab # [ 3.103907] systemd[1]: Mounting /sysroot/run...592gitlab # [ 3.107233] systemd[1]: Mounting /sysroot/tmp/shared...593gitlab # [ 3.120198] systemd[1]: Mounting /sysroot/tmp/xchg...594gitlab # [ 3.127303] systemd[1]: Starting Mountpoints Configured in the Real Root...595gitlab # [ 3.282121] 9p: Installing v9fs 9p2000 file system support596gitlab # [ 3.284104] erofs (device vdb): mounted with root inode @ nid 128.597gitlab # [ 3.138621] systemd[1]: Mounted /sysroot/nix/.rw-store.598gitlab # [ 3.142373] systemd[1]: Mounted /sysroot/nix/.ro-store.599gitlab # [ 3.144297] systemd[1]: Mounted /sysroot/run.600gitlab # [ 3.149365] systemd[1]: Mounted /sysroot/tmp/shared.601gitlab # [ 3.151257] systemd-sysroot-fstab-check[177]: /sysroot should be mounted in the initrd, will request daemon-reload.602gitlab # [ 3.157618] systemd[1]: Starting rw-sysroot-nix-store.service...603gitlab # [ 3.159517] systemd[1]: Mounted /sysroot/tmp/xchg.604gitlab # [ 3.161257] systemd[1]: Reload requested from client PID 177 ('systemd-sysroot') (unit initrd-parse-etc.service)...605gitlab # [ 3.164241] systemd[1]: Reloading...606gitlab # [ 3.219167] systemd[1]: Reloading finished in 59 ms.607gitlab # [ 3.238973] systemd-sysroot-fstab-check[177]: Requesting initrd-fs.target/start/replace...608gitlab # [ 3.242117] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.609gitlab # [ 3.244736] systemd[1]: Finished rw-sysroot-nix-store.service.610gitlab # [ 3.247130] systemd-sysroot-fstab-check[177]: Requesting swap.target/start/replace...611gitlab # [ 3.249772] systemd[1]: Starting rw-sysroot-nix-store.service...612gitlab # [ 3.251972] systemd[1]: initrd-parse-etc.service: Deactivated successfully.613gitlab # [ 3.254533] systemd[1]: Finished Mountpoints Configured in the Real Root.614gitlab # [ 3.256948] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.615gitlab # [ 3.267542] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.616gitlab # [ 3.270466] systemd[1]: Finished rw-sysroot-nix-store.service.617gitlab # [ 3.900387] systemd[1]: Mounting /sysroot/nix/store...618gitlab # [ 3.954759] systemd[1]: Mounted /sysroot/nix/store.619gitlab # [ 3.956725] systemd[1]: Reached target Initrd File Systems.620gitlab # [ 3.958764] systemd[1]: Starting Find NixOS closure...621gitlab # [ 3.960715] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...622gitlab # [ 3.987819] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.623gitlab # [ 3.992525] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.624gitlab # [ 4.005427] systemd[1]: Finished Find NixOS closure.625gitlab # [ 4.007338] systemd[1]: Reached target Initrd Default Target.626gitlab # [ 4.009833] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...627gitlab # [ 4.046374] systemd[1]: Stopped target Initrd Default Target.628gitlab # [ 4.048553] systemd[1]: Stopped target Basic System.629gitlab # [ 4.050458] systemd[1]: Stopped target Initrd Root Device.630gitlab # [ 4.052519] systemd[1]: Stopped target Path Units.631gitlab # [ 4.054246] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.632gitlab # [ 4.056746] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.633gitlab # [ 4.059409] systemd[1]: Stopped target Slice Units.634gitlab # [ 4.061246] systemd[1]: Stopped target Socket Units.635gitlab # [ 4.063152] systemd[1]: Stopped target System Initialization.636gitlab # [ 4.065527] systemd[1]: Stopped target Swaps.637gitlab # [ 4.068915] systemd[1]: Stopped target Timer Units.638gitlab # [ 4.070918] systemd[1]: dbus.socket: Deactivated successfully.639gitlab # [ 4.074809] systemd[1]: Closed D-Bus System Message Bus Socket.640gitlab # [ 4.077506] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.641gitlab # [ 4.081285] systemd[1]: Stopped Find NixOS closure.642gitlab # [ 4.083126] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio643gitlab # [ 4.086841] systemd[1]: Starting rw-sysroot-nix-store.service...644gitlab # [ 4.088925] systemd[1]: systemd-sysctl.service: Deactivated successfully.645gitlab # [ 4.091235] systemd[1]: Stopped Apply Kernel Variables.646gitlab # [ 4.093070] systemd[1]: systemd-modules-load.service: Deactivated successfully.647gitlab # [ 4.095485] systemd[1]: Stopped Load Kernel Modules.648gitlab # [ 4.096939] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.649gitlab # [ 4.099051] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.650gitlab # [ 4.101177] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.651gitlab # [ 4.103185] systemd[1]: Stopped Create System Files and Directories.652gitlab # [ 4.104905] systemd[1]: Stopped target Local File Systems.653gitlab # [ 4.106522] systemd[1]: Stopped target Preparation for Local File Systems.654gitlab # [ 4.108299] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.655gitlab # [ 4.109962] systemd[1]: Stopped Coldplug All udev Devices.656gitlab # [ 4.111198] systemd[1]: Stopping Rule-based Manager for Device Events and Files...657gitlab # [ 4.112844] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.658gitlab # [ 4.114576] systemd[1]: Stopped Virtual Console Setup.659gitlab # [ 4.115709] systemd[1]: initrd-cleanup.service: Deactivated successfully.660gitlab # [ 4.116806] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.661gitlab # [ 4.117784] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.662gitlab # [ 4.118800] systemd[1]: Finished rw-sysroot-nix-store.service.663gitlab # [ 4.119649] systemd[1]: systemd-udevd.service: Deactivated successfully.664gitlab # [ 4.120729] systemd[1]: Stopped Rule-based Manager for Device Events and Files.665gitlab # [ 4.121920] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.666gitlab # [ 4.122992] systemd[1]: Closed udev Control Socket.667gitlab # [ 4.123739] systemd[1]: Starting Cleanup udev Database...668gitlab # [ 4.124582] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.669gitlab # [ 4.125647] systemd[1]: Stopped Create Static Device Nodes in /dev.670gitlab # [ 4.126555] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.671gitlab # [ 4.127701] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.672gitlab # [ 4.128753] systemd[1]: kmod-static-nodes.service: Deactivated successfully.673gitlab # [ 4.129744] systemd[1]: Stopped Create List of Static Device Nodes.674gitlab # [ 4.138054] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.675gitlab # [ 4.139198] systemd[1]: Finished Cleanup udev Database.676gitlab # [ 4.140053] systemd[1]: Reached target Switch Root.677gitlab # [ 4.141658] systemd[1]: Starting NixOS Activation...678gitlab # [ 4.244495] initrd-nixos-activation-start[224]: booting system configuration /nix/store/1fhpq8fg97y6xiqyn6b5nfdx60qaisig-nixos-system-gitlab-test679gitlab # [ 4.292564] initrd-nixos-activation-start[224]: running activation script...680gitlab # [ 4.531125] initrd-nixos-activation-start[247]: setting up /etc...681gitlab # [ 4.643265] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.682gitlab # [ 4.644901] systemd[1]: Finished NixOS Activation.683gitlab # [ 4.647210] systemd[1]: Starting Switch Root...684gitlab # [ 4.668985] systemd[1]: Switching root.685gitlab # [ 4.885306] systemd-journald[92]: Received SIGTERM from PID 1 (systemd).686gitlab # [ 4.986319] NET: Registered PF_VSOCK protocol family687gitlab # [ 5.351945] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)688gitlab # [ 5.362087] systemd[1]: Detected virtualization kvm.689gitlab # [ 5.363983] systemd[1]: Detected architecture x86-64.690gitlab # [ 5.365920] systemd[1]: Detected first boot.691gitlab # [ 5.368443] systemd[1]: Initializing machine ID from random generator.692gitlab # [ 5.512311] systemd[1]: bpf-restrict-fs: LSM BPF program attached693gitlab # [ 5.610670] systemd[1]: Applying preset policy.694gitlab # [ 5.750047] systemd[1]: Populated /etc with preset unit settings.695gitlab # [ 5.869814] systemd[1]: initrd-switch-root.service: Deactivated successfully.696gitlab # [ 5.871253] systemd[1]: Stopped initrd-switch-root.service.697gitlab # [ 5.873733] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.698gitlab # [ 5.875897] systemd[1]: Created slice Slice /system/getty.699gitlab # [ 5.877307] systemd[1]: Created slice GitLab DevOps Platform Slice.700gitlab # [ 5.879096] systemd[1]: Created slice User and Session Slice.701gitlab # [ 5.880129] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.702gitlab # [ 5.881452] systemd[1]: Started Forward Password Requests to Wall Directory Watch.703gitlab # [ 5.882678] systemd[1]: Expecting device /dev/hvc0...704gitlab # [ 5.883469] systemd[1]: Expecting device /dev/ttyS0...705gitlab # [ 5.884257] systemd[1]: Reached target Local Encrypted Volumes.706gitlab # [ 5.885135] systemd[1]: Stopped target initrd-fs.target.707gitlab # [ 5.885955] systemd[1]: Stopped target initrd-root-fs.target.708gitlab # [ 5.886801] systemd[1]: Stopped target initrd-switch-root.target.709gitlab # [ 5.887692] systemd[1]: Reached target Virtual Machines and Containers.710gitlab # [ 5.888683] systemd[1]: Reached target Path Units.711gitlab # [ 5.889465] systemd[1]: Reached target Remote File Systems.712gitlab # [ 5.890293] systemd[1]: Reached target Slice Units.713gitlab # [ 5.891020] systemd[1]: Reached target Swaps.714gitlab # [ 5.892518] systemd[1]: Listening on Query the User Interactively for a Password.715gitlab # [ 5.894654] systemd[1]: Listening on Process Core Dump Socket.716gitlab # [ 5.896200] systemd[1]: Listening on Credential Encryption/Decryption.717gitlab # [ 5.897985] systemd[1]: Listening on Factory Reset Management.718gitlab # [ 5.898992] systemd[1]: Listening on Hostname Service Socket.719gitlab # [ 5.901433] systemd[1]: Starting Journal Log Access Socket...720gitlab # [ 5.902897] systemd[1]: Listening on Journal Audit Socket.721gitlab # [ 5.904881] systemd[1]: Listening on Console Output Muting Service Socket.722gitlab # [ 5.906090] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.723gitlab # [ 5.907317] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os724gitlab # [ 5.908814] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki725gitlab # [ 5.911779] systemd[1]: Listening on Disk Repartitioning Service Socket.726gitlab # [ 5.912935] systemd[1]: Listening on udev Control Socket.727gitlab # [ 5.913892] systemd[1]: Listening on udev Varlink Socket.728gitlab # [ 5.916742] systemd[1]: Mounting Huge Pages File System...729gitlab # [ 5.918890] systemd[1]: Mounting POSIX Message Queue File System...730gitlab # [ 5.920932] systemd[1]: Mounting Kernel Debug File System...731gitlab # [ 5.924842] systemd[1]: Mounting Kernel Trace File System...732gitlab # [ 5.931376] systemd[1]: Starting Create List of Static Device Nodes...733gitlab # [ 5.934146] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio734gitlab # [ 5.945765] systemd[1]: Starting Load Kernel Module configfs...735gitlab # [ 5.948218] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm736gitlab # [ 5.952055] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore737gitlab # [ 5.955996] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse738gitlab # [ 5.992499] systemd[1]: Mounting FUSE Control File System...739gitlab # [ 5.994313] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67740gitlab # [ 6.002164] systemd[1]: Starting Journal Service...741gitlab # [ 6.004341] systemd[1]: Starting Load Kernel Modules...742gitlab # [ 6.011508] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...743gitlab # [ 6.015636] systemd[1]: Starting Remount Root and Kernel File Systems...744gitlab # [ 6.016706] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os745gitlab # [ 6.023722] systemd[1]: Starting Coldplug All udev Devices...746gitlab # [ 6.028006] systemd[1]: Listening on Journal Log Access Socket.747gitlab # [ 6.029236] systemd[1]: Mounted Huge Pages File System.748gitlab # [ 6.030631] systemd[1]: Mounted POSIX Message Queue File System.749gitlab # [ 6.031852] systemd[1]: Mounted Kernel Debug File System.750gitlab # [ 6.032938] systemd[1]: Mounted Kernel Trace File System.751gitlab # [ 6.034380] systemd[1]: Finished Create List of Static Device Nodes.752gitlab # [ 6.035840] systemd[1]: modprobe@configfs.service: Deactivated successfully.753gitlab # [ 6.035922] EXT4-fs (vda): re-mounted 6e91375e-6b75-4e91-929f-1282cc2cdd2f.754gitlab # [ 6.036523] systemd[1]: Finished Load Kernel Module configfs.755gitlab # [ 6.041065] systemd[1]: Mounted FUSE Control File System.756gitlab # [ 6.041892] systemd-journald[318]: Collecting audit messages is enabled.757gitlab # [ 6.043290] systemd[1]: Finished Remount Root and Kernel File Systems.758gitlab # [ 6.045506] systemd[1]: Listening on Disk Image Download Service Socket.759gitlab # [ 6.048661] systemd[1]: Mounting Kernel Configuration File System...760gitlab # [ 6.050654] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore761gitlab # [ 6.057115] loop: module loaded762gitlab # [ 6.059642] systemd[1]: Starting Load/Save OS Random Seed...763gitlab # [ 5.914655] systemd[1]: Queued start job for default target Multi-User System.764gitlab # [ 5.915986] systemd[1]: Unnecessary job was removed for /dev/vdb.765gitlab # [ 5.917225] systemd[1]: systemd-journald.service: Deactivated successfully.766gitlab # [ 6.063657] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...767gitlab # [ 6.065887] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os768gitlab # [ 5.919792] systemd-modules-load[319]: Using 3 probe threads769gitlab # [ 5.922130] systemd-oomd[321]: No swap; memory pressure usage will be degraded770gitlab # [ 5.923751] systemd-modules-load[319]: Inserted module 'tls'771gitlab # [ 5.924775] systemd-modules-load[319]: Inserted module 'loop'772gitlab # [ 6.071210] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.773gitlab # [ 6.074253] systemd[1]: Started Journal Service.774gitlab # [ 5.930093] systemd[1]: Finished Load Kernel Modules.775gitlab # [ 5.932242] systemd[1]: Mounted Kernel Configuration File System.776gitlab # [ 5.937456] systemd[1]: Finished Load/Save OS Random Seed.777gitlab # [ 5.939636] systemd[1]: Reached target First Boot Complete.778gitlab # [ 5.941879] systemd[1]: Starting Firewall...779gitlab # [ 5.943457] systemd[1]: Starting Flush Journal to Persistent Storage...780gitlab # [ 5.966727] systemd[1]: Starting Apply Kernel Variables...781gitlab # [ 5.968039] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.782gitlab # [ 6.116619] systemd-journald[318]: Received client request to flush runtime journal.783gitlab # [ 6.002335] systemd[1]: Starting Create Static Device Nodes in /dev...784gitlab # [ 6.003940] systemd[1]: Finished Create Static Device Nodes in /dev.785gitlab # [ 6.005020] systemd[1]: Finished Apply Kernel Variables.786gitlab # [ 6.006062] systemd[1]: Reached target Preparation for Local File Systems.787gitlab # [ 6.007260] systemd[1]: Starting Rule-based Manager for Device Events and Files...788gitlab # [ 6.008529] systemd[1]: Finished Flush Journal to Persistent Storage.789gitlab # [ 6.022999] systemd-udevd[352]: Using default interface naming scheme 'v261'.790gitlab # [ 6.033888] systemd[1]: Finished Coldplug All udev Devices.791gitlab # [ 6.053023] systemd[1]: Started Rule-based Manager for Device Events and Files.792gitlab # [ 6.118155] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse793gitlab # [ 6.164466] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.794gitlab # [ 6.195259] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.795gitlab # [ 6.203124] (udev-worker)[432]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.796gitlab # [ 6.205341] (udev-worker)[432]: Network interface NamePolicy= disabled on kernel command line.797gitlab # [ 6.209851] (udev-worker)[423]: Network interface NamePolicy= disabled on kernel command line.798gitlab # [ 6.358159] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3799gitlab # [ 6.364667] ACPI: button: Power Button [PWRF]800gitlab # [ 6.241805] systemd[1]: Condition check resulted in Virtio network device being skipped.801gitlab # [ 6.395894] rtc_cmos 00:05: RTC can wake from S4802gitlab # [ 6.402042] rtc_cmos 00:05: registered as rtc0803gitlab # [ 6.402865] rtc_cmos 00:05: setting system clock to 2026-10-10T10:58:27 UTC (1791629907)804gitlab # [ 6.407859] parport_pc 00:03: reported by Plug and Play ACPI805gitlab # [ 6.409737] Floppy drive(s): fd0 is 2.88M AMI BIOS806gitlab # [ 6.410984] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs807gitlab # [ 6.413203] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]808gitlab # [ 6.413309] systemd-journald[318]: Time jumped backwards, rotating.809gitlab # [ 6.422147] FDC 0 is a S82078B810gitlab # [ 6.297076] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore811gitlab # [ 6.448240] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console812gitlab # [ 6.449278] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0813gitlab # [ 6.308469] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67814gitlab # [ 6.312550] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore815gitlab # [ 6.314682] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os816gitlab # [ 6.316315] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os817gitlab # [ 6.318424] systemd[1]: Finished Firewall.818gitlab # [ 6.451357] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5819gitlab # [ 6.456022] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD820gitlab # [ 6.473829] Console: switching to colour dummy device 80x25821gitlab # [ 6.476989] [drm] Found bochs VGA, ID 0xb0c5.822gitlab # [ 6.476991] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.823gitlab # [ 6.479524] mousedev: PS/2 mouse device common for all mice824gitlab # [ 6.481696] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic825gitlab # [ 6.482485] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0826gitlab # [ 6.487991] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4827gitlab # [ 6.491005] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6828gitlab # [ 6.510491] Console: switching to colour frame buffer device 160x50829gitlab # [ 6.515423] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device830gitlab # [ 6.379462] systemd[1]: Starting Virtual Console Setup...831gitlab # [ 6.537266] ppdev: user-space parallel port driver832gitlab # [ 6.406967] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.833gitlab # [ 6.408170] systemd[1]: Stopped Virtual Console Setup.834gitlab # [ 6.411216] systemd[1]: Starting Virtual Console Setup...835gitlab # [ 6.420658] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.836gitlab # [ 6.421780] systemd[1]: Stopped Virtual Console Setup.837gitlab # [ 6.441669] systemd[1]: Starting Virtual Console Setup...838gitlab # [ 6.634185] kvm_amd: TSC scaling supported839gitlab # [ 6.634809] kvm_amd: Nested Virtualization enabled840gitlab # [ 6.635413] kvm_amd: Nested Paging enabled841gitlab # [ 6.636037] kvm_amd: LBR virtualization supported842gitlab # [ 6.636695] kvm_amd: Virtual VMLOAD VMSAVE supported843gitlab # [ 6.637326] kvm_amd: Virtual GIF supported844gitlab # [ 6.637930] kvm_amd: Virtual NMI enabled845gitlab # [ 6.670230] EDAC MC: Ver: 3.0.0846gitlab # [ 6.524398] systemd-vconsole-setup[495]: Configuration of first virtual console was skipped, ignoring remaining ones.847gitlab # [ 6.527604] systemd[1]: Finished Virtual Console Setup.848gitlab # [ 6.728420] systemd[1]: Mounting /run/wrappers...849gitlab # [ 6.751098] systemd[1]: Mounted /run/wrappers.850gitlab # [ 6.752737] systemd[1]: Reached target Local File Systems.851gitlab # [ 6.754345] systemd[1]: Listening on Boot Loader Control Service Socket.852gitlab # [ 6.756842] systemd[1]: Starting register-nix-paths.service...853gitlab # [ 6.759594] systemd[1]: Starting Create SUID/SGID Wrappers...854gitlab # [ 6.761059] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.855gitlab # [ 6.762747] systemd[1]: Starting Save Transient machine-id to Disk...856gitlab # [ 6.770222] systemd[1]: Starting Create System Files and Directories...857gitlab # [ 6.793593] systemd[1]: Finished Create System Files and Directories.858gitlab # [ 6.796964] systemd[1]: Starting Rebuild Journal Catalog...859gitlab # [ 6.799278] systemd[1]: Starting Record System Boot/Shutdown in UTMP...860gitlab # [ 6.814465] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.861gitlab # [ 6.816371] systemd[1]: Finished Save Transient machine-id to Disk.862gitlab # [ 6.822870] systemd[1]: Finished Record System Boot/Shutdown in UTMP.863gitlab # [ 6.835248] systemd[1]: Finished Rebuild Journal Catalog.864gitlab # [ 6.837458] systemd[1]: Starting Update is Completed...865gitlab # [ 6.852091] systemd[1]: Finished Update is Completed.866gitlab # [ 6.917373] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.867gitlab # [ 6.918831] systemd[1]: Finished Create SUID/SGID Wrappers.868gitlab # [ 7.169483] systemd[1]: Finished register-nix-paths.service.869gitlab # [ 7.171725] systemd[1]: Reached target System Initialization.870gitlab # [ 7.173693] systemd[1]: Started Discard unused filesystem blocks once a week.871gitlab # [ 7.176162] systemd[1]: Started logrotate.timer.872gitlab # [ 7.177873] systemd[1]: Started Daily Cleanup of Temporary Directories.873gitlab # [ 7.180193] systemd[1]: Reached target Timer Units.874gitlab # [ 7.181969] systemd[1]: Listening on D-Bus System Message Bus Socket.875gitlab # [ 7.184231] systemd[1]: Listening on Nix Daemon Socket.876gitlab # [ 7.186417] systemd[1]: Starting nixbot CI service socket...877gitlab # [ 7.188727] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.878gitlab # [ 7.197166] systemd[1]: Starting D-Bus System Message Bus...879gitlab # [ 7.199547] systemd[1]: Listening on nixbot CI service socket.880gitlab # [ 7.201502] systemd[1]: Reached target Socket Units.881gitlab # [ 7.232976] dbus-broker-launch[577]: Looking up NSS user entry for 'systemd-timesync'...882gitlab # [ 7.236336] dbus-broker-launch[577]: NSS returned no entry for 'systemd-timesync'883gitlab # [ 7.239084] dbus-broker-launch[577]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"884gitlab # [ 7.244235] systemd[1]: Started D-Bus System Message Bus.885gitlab # [ 7.256084] dbus-broker-launch[577]: Ready886gitlab # [ 7.258361] systemd[1]: Reached target Basic System.887gitlab # [ 7.261337] systemd[1]: Started backdoor.service.888gitlab # [ 7.266185] systemd[1]: Starting gitlab-config.service...889gitlab # [ 7.268868] systemd[1]: Starting Import lastlog data into lastlog2 database...890gitlab # [ 7.280503] systemd[1]: Starting Logrotate configuration check...891gitlab # [ 7.285900] systemd[1]: Starting Name Service Cache Daemon (nsncd)...892gitlab # [ 7.295286] systemd[1]: Starting Post-Boot Actions...893gitlab # connecting to host...894gitlab # [ 7.301596] logrotate[582]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.895gitlab # [ 7.303541] logrotate[582]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf896gitlab # [ 7.305181] logrotate[582]: note: 'monthly' overrides previously specified 'weekly'897gitlab # [ 7.306125] logrotate[582]: Reading state from file: /var/lib/logrotate.status898gitlab # [ 7.307238] logrotate[582]: state file /var/lib/logrotate.status does not exist899gitlab # [ 7.308469] logrotate[582]: Allocating hash table for state file, size 64 entries900gitlab # [ 7.309655] logrotate[582]: Handling 4 logs901gitlab # [ 7.310740] logrotate[582]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed902gitlab # [ 7.311846] logrotate[582]: considering log /var/log/btmp903gitlab # [ 7.312825] logrotate[582]: Creating new state904gitlab # [ 7.313742] logrotate[582]: Now: 2026-10-10 10:58905gitlab # [ 7.314674] logrotate[582]: Last rotated at 2026-10-10 10:00906gitlab # [ 7.315597] logrotate[582]: log does not need rotating (log has already been rotated)907gitlab # [ 7.315918] logrotate[582]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed908gitlab # [ 7.316854] logrotate[582]: considering log /var/log/wtmp909gitlab # [ 7.317722] logrotate[582]: Creating new state910gitlab # [ 7.318366] logrotate[582]: Now: 2026-10-10 10:58911gitlab # [ 7.318953] logrotate[582]: Last rotated at 2026-10-10 10:00912gitlab # [ 7.319894] logrotate[582]: log does not need rotating (log has already been rotated)913gitlab # [ 7.320792] logrotate[582]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed914gitlab # [ 7.321723] logrotate[582]: switching euid from 0 to 165 and egid from 0 to 165 (pid 582)915gitlab # [ 7.322638] logrotate[582]: considering log /var/gitlab/state/log/*.log916gitlab # [ 7.323590] logrotate[582]: log /var/gitlab/state/log/*.log does not exist -- skipping917gitlab # [ 7.324532] logrotate[582]: Creating new state918gitlab # [ 7.325437] logrotate[582]: switching euid from 165 to 0 and egid from 165 to 0 (pid 582)919gitlab # [ 7.326367] logrotate[582]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed920gitlab # [ 7.327295] logrotate[582]: switching euid from 0 to 60 and egid from 0 to 60 (pid 582)921gitlab # [ 7.327885] logrotate[582]: considering log /var/log/nginx/*.log922gitlab # [ 7.328806] logrotate[582]: log /var/log/nginx/*.log does not exist -- skipping923gitlab # [ 7.329714] logrotate[582]: Creating new state924gitlab # [ 7.330631] logrotate[582]: not running postrotate script, since no logs were rotated925gitlab # [ 7.331531] logrotate[582]: switching euid from 60 to 0 and egid from 60 to 0 (pid 582)926gitlab # [ 7.349252] systemd[1]: Started Reset console on configuration changes.927gitlab # [ 7.354202] systemd[1]: Starting resolvconf update...928gitlab: Guest shell says: b'Spawning backdoor root shell...\n'929gitlab: connected to guest root shell930gitlab: (connecting took 8.00 seconds)931gitlab: (finished: waiting for the VM to finish booting, in 8.00 seconds)932gitlab # [ 7.368992] systemd[1]: Finished Logrotate configuration check.933gitlab # [ 7.371172] systemd[1]: Finished Post-Boot Actions.934gitlab # [ 7.377887] nsncd[583]: Oct 10 10:58:28.621 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"935gitlab # [ 7.378586] systemd[1]: Started Name Service Cache Daemon (nsncd).936gitlab # [ 7.381175] systemd[1]: Finished Import lastlog data into lastlog2 database.937gitlab # [ 7.386121] systemd[1]: Reached target Host and Network Name Lookups.938gitlab # [ 7.387222] systemd[1]: Reached target User and Group Name Lookups.939gitlab # [ 7.389588] systemd[1]: Starting User Login Management...940gitlab # [ 7.431796] systemd-logind[622]: New seat seat0.941gitlab # [ 7.434289] systemd-logind[622]: Watching system buttons on /dev/input/event2 (Power Button)942gitlab # [ 7.434933] systemd-logind[622]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard)943gitlab # [ 7.435564] systemd-logind[622]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)944gitlab # [ 7.438492] systemd[1]: Started User Login Management.945gitlab # [ 7.442759] systemd[1]: Stopped target Host and Network Name Lookups.946gitlab # [ 7.443573] systemd[1]: Stopping Host and Network Name Lookups...947gitlab # [ 7.444511] systemd[1]: Stopped target User and Group Name Lookups.948gitlab # [ 7.446031] systemd[1]: Stopping User and Group Name Lookups...949gitlab # [ 7.447912] systemd[1]: Starting linger-users.service...950gitlab # [ 7.449606] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...951gitlab # [ 7.450414] systemd[1]: nscd.service: Deactivated successfully.952gitlab # [ 7.451111] systemd[1]: Stopped Name Service Cache Daemon (nsncd).953gitlab # [ 7.451630] systemd[1]: nscd.service: Consumed 25ms CPU time over 160ms wall clock time, 7M memory peak, 5.5M read from disk.954gitlab # [ 7.455911] systemd[1]: Starting Name Service Cache Daemon (nsncd)...955gitlab # [ 7.464521] systemd[1]: linger-users.service: Deactivated successfully.956gitlab # [ 7.465287] systemd[1]: Finished linger-users.service.957gitlab # [ 7.470662] nsncd[677]: Oct 10 10:58:28.714 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"958gitlab # [ 7.472673] systemd[1]: Started Name Service Cache Daemon (nsncd).959gitlab # [ 7.473641] systemd[1]: Reached target Host and Network Name Lookups.960gitlab # [ 7.474743] systemd[1]: Reached target User and Group Name Lookups.961gitlab # [ 7.477350] systemd[1]: Finished resolvconf update.962gitlab # [ 7.478429] systemd[1]: Reached target Preparation for Network.963gitlab # [ 7.480679] systemd[1]: Starting DHCP Client...964gitlab # [ 7.481465] systemd[1]: Starting Address configuration of eth1...965gitlab # [ 7.483700] systemd[1]: Starting Extra networking commands....966gitlab # [ 7.506271] network-addresses-eth1-start[700]: adding address 192.168.1.1/24... done967gitlab # [ 7.508459] network-addresses-eth1-start[700]: adding address 2001:db8:1::1/64... done968gitlab # [ 7.513074] systemd[1]: Finished Address configuration of eth1.969gitlab # [ 7.542614] systemd[1]: Finished Extra networking commands..970gitlab # [ 7.543620] systemd[1]: Reached target Network.971gitlab # [ 7.544515] dhcpcd[709]: dhcpcd-10.3.2 starting972gitlab # [ 7.545476] systemd[1]: Starting gitlab-workhorse.service...973gitlab # [ 7.546551] dhcpcd[766]: dev: loaded udev974gitlab # [ 7.548406] systemd[1]: Starting Nginx Web Server...975gitlab # [ 7.552158] s[ 7.698792] 8021q: 802.1Q VLAN Support v1.8976gitlab # ystemd[1]: Start[ 7.699441] 8021q: adding VLAN 0 to HW filter on device eth1977gitlab # ing PostgreSQL Server...978gitlab # [ 7.555598] systemd[1]: Starting Redis Server - redis-gitlab...979gitlab # [ 7.559417] systemd[1]: Starting Permit User Sessions...980gitlab # [ 7.567067] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.981gitlab # [ 7.582419] systemd[1]: Finished Permit User Sessions.982gitlab # [ 7.585365] systemd[1]: Started Getty on tty1.983gitlab # [ 7.586314] systemd[1]: Reached target Login Prompts.984gitlab # [ 7.777000] cfg80211: Loading compiled-in X.509 certificates for regulatory database985gitlab # [ 7.782300] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'986gitlab # [ 7.783197] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'987gitlab # [ 7.784646] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2988gitlab # [ 7.785722] cfg80211: failed to load regulatory.db989gitlab # [ 7.819225] 8021q: adding VLAN 0 to HW filter on device eth0990gitlab # [ 7.676637] dhcpcd[766]: eth0: waiting for carrier991gitlab # [ 7.677569] dhcpcd[766]: eth0: carrier acquired992gitlab # [ 7.682896] dhcpcd[766]: DUID 00:01:00:01:32:5c:d6:d4:52:54:00:12:34:56993gitlab # [ 7.683927] dhcpcd[766]: eth0: IAID 00:12:34:56994gitlab # [ 7.684461] dhcpcd[766]: eth0: adding address fe80::5054:ff:fe12:3456995gitlab # [ 7.701093] nginx-pre-start[793]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok996gitlab # [ 7.701788] nginx-pre-start[793]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful997gitlab # [ 7.702815] redis-server[788]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.998gitlab # [ 7.703605] redis-server[788]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo999gitlab # [ 7.704194] redis-server[788]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=788, just started1000gitlab # [ 7.704313] redis-server[788]: Configuration loaded1001gitlab # [ 7.704767] redis[788]: monotonic clock: POSIX clock_gettime1002gitlab # [ 7.705519] redis[788]: Running mode=standalone, port=0.1003gitlab # [ 7.706133] systemd[1]: Started Redis Server - redis-gitlab.1004gitlab # [ 7.706677] redis[788]: Server initialized1005gitlab # [ 7.707220] redis[788]: Ready to accept connections unix1006gitlab # [ 7.707710] redis[788]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1007gitlab # [ 7.716249] postgresql-pre-start[795]: The files belonging to this database system will be owned by user "postgres".1008gitlab # [ 7.717054] postgresql-pre-start[795]: This user must also own the server process.1009gitlab # [ 7.721076] systemd[1]: Started Nginx Web Server.1010gitlab # [ 7.722372] postgresql-pre-start[795]: The database cluster will be initialized with locale "en_US.UTF-8".1011gitlab # [ 7.723944] postgresql-pre-start[795]: The default database encoding has accordingly been set to "UTF8".1012gitlab # [ 7.725529] postgresql-pre-start[795]: The default text search configuration will be set to "english".1013gitlab # [ 7.726938] postgresql-pre-start[795]: Data page checksums are enabled.1014gitlab # [ 7.728337] postgresql-pre-start[795]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok1015gitlab # [ 7.730326] postgresql-pre-start[795]: creating subdirectories ... ok1016gitlab # [ 7.731708] postgresql-pre-start[795]: selecting dynamic shared memory implementation ... posix1017gitlab # [ 7.774466] postgresql-pre-start[795]: selecting default "max_connections" ... 1001018gitlab # [ 7.820468] postgresql-pre-start[795]: selecting default "shared_buffers" ... 128MB1019gitlab # [ 8.054099] systemd[1]: Started gitlab-workhorse.service.1020gitlab # [ 8.203044] postgresql-pre-start[795]: selecting default time zone ... UTC1021gitlab # [ 8.205398] postgresql-pre-start[795]: creating configuration files ... ok1022gitlab # [ 8.249564] gitlab-workhorse[825]: time="2026-10-10T10:58:29Z" level=info msg=Starting build_time=19700101.000000 version=19.0.41023gitlab # [ 8.249969] gitlab-workhorse[825]: time="2026-10-10T10:58:29Z" level=info msg="Using redis/go-redis"1024gitlab # [ 8.250610] gitlab-workhorse[825]: time="2026-10-10T10:58:29Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1025gitlab # [ 8.380547] postgresql-pre-start[795]: running bootstrap script ... ok1026gitlab # [ 8.690666] dhcpcd[766]: eth0: soliciting a DHCP lease1027gitlab # [ 8.849299] NET: Registered PF_PACKET protocol family1028gitlab # [ 8.711931] dhcpcd[766]: eth0: offered 10.0.2.15 from 10.0.2.21029gitlab # [ 8.718403] dhcpcd[766]: eth0: probing address 10.0.2.15/241030gitlab # [ 8.785255] postgresql-pre-start[795]: performing post-bootstrap initialization ... ok1031gitlab # [ 9.080217] postgresql-pre-start[795]: syncing data to disk ... ok1032gitlab # [ 9.082431] postgresql-pre-start[795]: initdb: warning: enabling "trust" authentication for local connections1033gitlab # [ 9.084273] postgresql-pre-start[795]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1034gitlab # [ 9.085955] postgresql-pre-start[795]: Success. You can now start the database server using:1035gitlab # [ 9.088157] postgresql-pre-start[795]: pg_ctl -D /var/lib/postgresql/18 -l logfile start1036gitlab # [ 9.205629] postgres[851]: [851] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1037gitlab # [ 9.210234] postgres[851]: [851] LOG: listening on IPv6 address "::1", port 54321038gitlab # [ 9.213259] postgres[851]: [851] LOG: listening on IPv4 address "127.0.0.1", port 54321039gitlab # [ 9.216420] postgres[851]: [851] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1040gitlab # [ 9.227303] postgres[860]: [860] LOG: database system was shut down at 2026-10-10 10:58:30 GMT1041gitlab # [ 9.235501] postgres[851]: [851] LOG: database system is ready to accept connections1042gitlab # [ 9.240549] systemd[1]: Started PostgreSQL Server.1043gitlab # [ 9.247241] systemd[1]: Starting PostgreSQL Setup Scripts...1044gitlab # [ 9.395228] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[868]: mkdir -p /var/gitlab/state/home/.ssh: OK1045gitlab # [ 9.398734] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[868]: chmod 700 /var/gitlab/state/home/.ssh: OK1046gitlab # [ 9.410852] postgresql-setup-start[876]: CREATE DATABASE1047gitlab # [ 9.429062] postgresql-setup-start[888]: CREATE ROLE1048gitlab # [ 9.436920] postgresql-setup-start[892]: ALTER DATABASE1049gitlab # [ 9.452595] postgresql-setup-start[899]: CREATE ROLE1050gitlab # [ 9.454919] systemd[1]: Finished PostgreSQL Setup Scripts.1051gitlab # [ 9.457298] systemd[1]: Reached target PostgreSQL.1052gitlab # [ 9.459275] systemd[1]: Starting gitlab-postgresql.service...1053gitlab # [ 9.461640] systemd[1]: Finished gitlab-config.service.1054gitlab # [ 9.465398] systemd[1]: Started gitaly.service.1055gitlab # [ 9.467062] systemd[1]: Starting gitlab-db-config.service...1056gitlab # [ 9.554625] gitlab-postgresql-start[920]: CREATE DATABASE1057gitlab # [ 9.589161] gitlab-postgresql-start[926]: CREATE EXTENSION1058gitlab # [ 9.667997] gitlab-postgresql-start[928]: CREATE EXTENSION1059gitlab # [ 9.670774] systemd[1]: Finished gitlab-postgresql.service.1060gitlab # [ 9.985931] dhcpcd[766]: eth0: soliciting an IPv6 router1061gitlab # [ 9.987099] dhcpcd[766]: eth0: Router Advertisement from fe80::21062gitlab # [ 9.987935] dhcpcd[766]: eth0: adding address fec0::5054:ff:fe12:3456/641063gitlab # [ 9.989035] dhcpcd[766]: eth0: adding route to fec0::/641064gitlab # [ 9.989782] dhcpcd[766]: eth0: adding default route via fe80::21065gitlab # [ 9.999651] gitaly[904]: time="2026-10-10T10:58:31.242Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=9041066gitlab # [ 10.008694] gitaly[904]: time="2026-10-10T10:58:31.252Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=9041067gitlab # [ 10.009566] gitaly[904]: time="2026-10-10T10:58:31.252Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=9041068gitlab # [ 10.018311] gitaly[904]: time="2026-10-10T10:58:31.261Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=9041069gitlab # [ 10.019090] gitaly[904]: time="2026-10-10T10:58:31.261Z" level=info msg="Starting Gitaly" pid=904 version=1070gitlab # [ 10.187725] gitaly[904]: time="2026-10-10T10:58:31.431Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=160 pid=9041071gitlab # [ 10.189775] gitaly[904]: time="2026-10-10T10:58:31.432Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=9041072gitlab # [ 10.191577] gitaly[904]: time="2026-10-10T10:58:31.432Z" level=info msg="finished initializing command factory" duration_ms=0 pid=9041073gitlab # [ 10.193459] gitaly[904]: time="2026-10-10T10:58:31.433Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-904/git-exec-45044560.d/git pid=9041074gitlab # [ 10.195563] gitaly[904]: time="2026-10-10T10:58:31.436Z" level=info msg="finished detecting git version" duration_ms=3 pid=9041075gitlab # [ 10.197359] gitaly[904]: time="2026-10-10T10:58:31.436Z" level=info msg="using Git version" pid=904 version=2.53.g1076gitlab # [ 10.198978] gitaly[904]: time="2026-10-10T10:58:31.437Z" level=info msg="clearing disk cache object folder" pid=904 storage=default1077gitlab # [ 10.200828] gitaly[904]: time="2026-10-10T10:58:31.437Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3684927667 pid=904 storage=default1078gitlab # [ 10.203532] gitaly[904]: time="2026-10-10T10:58:31.437Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3684927667 pid=904 storage=default1079gitlab # [ 10.206638] gitaly[904]: time="2026-10-10T10:58:31.437Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=9041080gitlab # [ 10.209387] gitaly[904]: time="2026-10-10T10:58:31.437Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=9041081gitlab # [ 10.212038] gitaly[904]: time="2026-10-10T10:58:31.437Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3684927667 clear_duration_ms=0 pid=904 storage=default1082gitlab # [ 10.228129] gitaly[904]: time="2026-10-10T10:58:31.471Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9041083gitlab # [ 10.229080] gitaly[904]: time="2026-10-10T10:58:31.471Z" level=info msg="counting repositories" pid=9041084gitlab # [ 10.229478] gitaly[904]: time="2026-10-10T10:58:31.471Z" level=info msg="starting to count repositories" pid=904 storage_path=/var/gitlab/state/repositories1085gitlab # [ 10.230051] gitaly[904]: time="2026-10-10T10:58:31.471Z" level=info msg="finished tempdir cleaner walk" pid=904 storage=default time_ms=01086gitlab # [ 10.230479] gitaly[904]: time="2026-10-10T10:58:31.471Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=904 schema=unix1087gitlab # [ 10.230995] gitaly[904]: time="2026-10-10T10:58:31.472Z" level=info msg="listening at address" address=/run/gitaly/gitaly-904/sock.d/intern pid=904 schema=unix1088gitlab # [ 10.238057] gitaly[904]: time="2026-10-10T10:58:31.481Z" level=info msg="completed counting repositories" counting_duration_ms=9 pid=904 storage_path=/var/gitlab/state/repositories1089gitlab # [ 10.238646] gitaly[904]: time="2026-10-10T10:58:31.481Z" level=info msg="completed counting all repositories" counting_duration=9 pid=9041090gitlab # [ 10.239255] gitaly[904]: time="2026-10-10T10:58:31.482Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9041091gitlab # [ 10.239816] gitaly[904]: time="2026-10-10T10:58:31.482Z" level=info msg="Started Gitaly" duration_ms=220 pid=9041092gitlab # [ 10.240447] gitaly[904]: time="2026-10-10T10:58:31.482Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9041093gitlab # [ 10.240963] gitaly[904]: time="2026-10-10T10:58:31.483Z" level=info msg="maintenance: daily scheduled" pid=904 scheduled="2026-10-10 12:00:00 +0000 UTC"1094gitlab # [ 10.848473] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[911]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1095gitlab # [ 10.850831] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[911]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1096gitlab # [ 10.871589] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[911]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1097gitlab # [ 12.119680] gitaly[904]: time="2026-10-10T10:58:33.362Z" level=info msg="License database preloaded" duration_ms=2100 pid=9041098gitlab # [ 13.721269] dhcpcd[766]: eth0: leased 10.0.2.15 for 86400 seconds1099gitlab # [ 13.723467] dhcpcd[766]: eth0: adding route to 10.0.2.0/241100gitlab # [ 13.725316] dhcpcd[766]: eth0: adding default route via 10.0.2.21101gitlab # [ 13.844600] systemd[1]: Started DHCP Client.1102gitlab # [ 13.846902] systemd[1]: Reached target Network is Online.1103gitlab # [ 40.693982] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[911]: W, [2026-10-10T10:59:01.936829 #911] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1104gitlab # [ 42.474759] postgres[1307]: [1307] ERROR: relation "feature_gates" does not exist at character 5231105gitlab # [ 42.475514] postgres[1307]: [1307] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1106gitlab # [ 42.476353] postgres[1307]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1107gitlab # [ 42.477280] postgres[1307]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1108gitlab # [ 42.478475] postgres[1307]: attidentity AS identity,1109gitlab # [ 42.480053] postgres[1307]: attgenerated as attgenerated1110gitlab # [ 42.481174] postgres[1307]: FROM pg_attribute a1111gitlab # [ 42.482307] postgres[1307]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1112gitlab # [ 42.483434] postgres[1307]: LEFT JOIN pg_type t ON a.atttypid = t.oid1113gitlab # [ 42.484929] postgres[1307]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1114gitlab # [ 42.486071] postgres[1307]: WHERE a.attrelid = '"feature_gates"'::regclass1115gitlab # [ 42.487179] postgres[1307]: AND a.attnum > 0 AND NOT a.attisdropped1116gitlab # [ 42.488290] postgres[1307]: ORDER BY a.attnum1117gitlab # [ 42.489417] postgres[1307]:1118gitlab # [ 46.216929] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1340]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1119gitlab # [ 46.218735] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1340]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1120gitlab # [ 48.277129] gitlab-workhorse[825]: time="2026-10-10T10:59:09Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1121gitlab # [ 58.287545] gitlab-workhorse[825]: time="2026-10-10T10:59:19Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1122gitlab # [ 61.151507] postgres[1415]: [1415] ERROR: relation "table_without_models" does not exist1123gitlab # [ 61.153374] postgres[1415]: [1415] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1124gitlab # [ 68.288841] gitlab-workhorse[825]: time="2026-10-10T10:59:29Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1125gitlab # [ 73.219274] systemd[1]: Finished gitlab-db-config.service.1126gitlab # [ 73.224173] systemd[1]: Started gitlab-sidekiq.service.1127gitlab # [ 73.227328] systemd[1]: Starting gitlab.service...1128gitlab # [ 73.578255] bundle[1582]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1129gitlab # [ 73.579543] bundle[1582]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1130gitlab # [ 73.580407] bundle[1582]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1131gitlab # [ 76.251685] sidekiq-cluster[1581]: {"severity":"INFO","time":"2026-10-10T10:59:37.478Z","message":"Starting cluster with 1 processes"}1132gitlab # [ 76.347156] bundle[1582]: {"timestamp":"2026-10-10T10:59:37.570Z","pid":1582,"message":"* Enabling systemd notification integration"}1133gitlab # [ 76.349262] bundle[1582]: {"timestamp":"2026-10-10T10:59:37.590Z","pid":1582,"message":"Puma starting in cluster mode..."}1134gitlab # [ 76.350527] bundle[1582]: {"timestamp":"2026-10-10T10:59:37.590Z","pid":1582,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1135gitlab # [ 76.352120] bundle[1582]: {"timestamp":"2026-10-10T10:59:37.590Z","pid":1582,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1136gitlab # [ 76.353291] bundle[1582]: {"timestamp":"2026-10-10T10:59:37.590Z","pid":1582,"message":"* Min threads: 0"}1137gitlab # [ 76.354514] bundle[1582]: {"timestamp":"2026-10-10T10:59:37.590Z","pid":1582,"message":"* Max threads: 4"}1138gitlab # [ 76.356202] bundle[1582]: {"timestamp":"2026-10-10T10:59:37.590Z","pid":1582,"message":"* Environment: production"}1139gitlab # [ 76.357346] bundle[1582]: {"timestamp":"2026-10-10T10:59:37.590Z","pid":1582,"message":"* Master PID: 1582"}1140gitlab # [ 76.359044] bundle[1582]: {"timestamp":"2026-10-10T10:59:37.590Z","pid":1582,"message":"* Workers: 2"}1141gitlab # [ 76.360332] bundle[1582]: {"timestamp":"2026-10-10T10:59:37.590Z","pid":1582,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1142gitlab # [ 76.361861] bundle[1582]: {"timestamp":"2026-10-10T10:59:37.590Z","pid":1582,"message":"* Preloading application"}1143gitlab # [ 78.298923] gitlab-workhorse[825]: time="2026-10-10T10:59:39Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1144gitlab # [ 88.308846] gitlab-workhorse[825]: time="2026-10-10T10:59:49Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1145gitlab # [ 98.318942] gitlab-workhorse[825]: time="2026-10-10T10:59:59Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1146gitlab # [ 98.853513] bundle[1582]: W, [2026-10-10T11:00:00.096260 #1582] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1147gitlab # [ 99.219594] systemd[1]: Starting Logrotate Service...1148gitlab # [ 99.296281] systemd[1]: logrotate.service: Deactivated successfully.1149gitlab # [ 99.298059] systemd[1]: Finished Logrotate Service.1150gitlab # [ 101.307798] sidekiq-cluster[1612]: 2026-10-10T11:00:02.550Z pid=1612 tid=wkk INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1612, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>433938432}1151gitlab # [ 101.479638] sidekiq-cluster[1612]: W, [2026-10-10T11:00:02.722999 #1612] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1152gitlab # [ 101.518729] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:02.762Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1153gitlab # [ 103.365362] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.607Z","message":"GitLab reliable fetch activated!"}1154gitlab # [ 103.367710] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.608Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1155gitlab # [ 103.387819] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.631Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1156gitlab # [ 103.395382] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.638Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1157gitlab # [ 103.404807] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.648Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1158gitlab # [ 103.413917] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.657Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1159gitlab # [ 103.418473] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.662Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1160gitlab # [ 103.422685] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.666Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1161gitlab # [ 103.425882] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.669Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1162gitlab # [ 103.429114] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.672Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1163gitlab # [ 103.432369] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.675Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1164gitlab # [ 103.435676] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.679Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1165gitlab # [ 103.439142] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.682Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1166gitlab # [ 103.443704] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.687Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1167gitlab # [ 103.447227] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.690Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1168gitlab # [ 103.450221] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.693Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1169gitlab # [ 103.453162] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.696Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1170gitlab # [ 103.456145] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.699Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1171gitlab # [ 103.460103] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.703Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1172gitlab # [ 103.463570] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.707Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1173gitlab # [ 103.467234] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.710Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1174gitlab # [ 103.470756] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.714Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1175gitlab # [ 103.474324] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.717Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1176gitlab # [ 103.476858] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.720Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1177gitlab # [ 103.478861] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.722Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1178gitlab # [ 103.480751] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.724Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1179gitlab # [ 103.482430] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.726Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1180gitlab # [ 103.484373] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.727Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1181gitlab # [ 103.486237] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.729Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1182gitlab # [ 103.488082] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.731Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1183gitlab # [ 103.490058] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.733Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1184gitlab # [ 103.491974] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.735Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1185gitlab # [ 103.493543] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.737Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1186gitlab # [ 103.495443] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.739Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1187gitlab # [ 103.497437] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.741Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1188gitlab # [ 103.499470] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.743Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1189gitlab # [ 103.501418] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.745Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1190gitlab # [ 103.503297] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.746Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1191gitlab # [ 103.504939] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.748Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1192gitlab # [ 103.507333] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.750Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1193gitlab # [ 103.509389] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.753Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1194gitlab # [ 103.511483] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.755Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1195gitlab # [ 103.514099] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.757Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1196gitlab # [ 103.517690] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.761Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1197gitlab # [ 103.521303] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.764Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1198gitlab # [ 103.525143] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.768Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1199gitlab # [ 103.529136] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.772Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1200gitlab # [ 103.532883] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.776Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1201gitlab # [ 103.536286] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.779Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1202gitlab # [ 103.541377] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.784Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1203gitlab # [ 103.545419] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.789Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1204gitlab # [ 103.547893] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.791Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1205gitlab # [ 103.550318] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.793Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1206gitlab # [ 103.553099] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.796Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1207gitlab # [ 103.562531] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.806Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1208gitlab # [ 103.565503] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.809Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1209gitlab # [ 103.569811] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.813Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1210gitlab # [ 103.574042] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.817Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1211gitlab # [ 103.577970] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.821Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1212gitlab # [ 103.582155] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.825Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1213gitlab # [ 103.585817] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.829Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1214gitlab # [ 103.589590] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.833Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1215gitlab # [ 103.593723] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.837Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1216gitlab # [ 103.599630] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.843Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1217gitlab # [ 103.605671] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.849Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1218gitlab # [ 103.611469] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.855Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1219gitlab # [ 103.616799] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.860Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1220gitlab # [ 103.622688] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.866Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1221gitlab # [ 103.626973] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.870Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1222gitlab # [ 103.631858] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.875Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1223gitlab # [ 103.636639] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.880Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1224gitlab # [ 103.641565] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.885Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1225gitlab # [ 103.679546] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.922Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1226gitlab # [ 103.687672] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.931Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1227gitlab # [ 103.695222] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.938Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1228gitlab # [ 103.702643] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.946Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1229gitlab # [ 103.711294] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.954Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1230gitlab # [ 103.718462] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.962Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1231gitlab # [ 103.727475] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.970Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1232gitlab # [ 103.734911] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.978Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1233gitlab # [ 103.742340] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.985Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1234gitlab # [ 103.748953] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.992Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1235gitlab # [ 103.756062] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:04.999Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1236gitlab # [ 103.763758] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.007Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1237gitlab # [ 103.773930] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.017Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1238gitlab # [ 103.783260] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.026Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1239gitlab # [ 103.791636] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.035Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1240gitlab # [ 103.800100] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.043Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1241gitlab # [ 103.808891] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.052Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1242gitlab # [ 103.817549] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.061Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1243gitlab # [ 103.825526] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.069Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1244gitlab # [ 103.834286] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.077Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1245gitlab # [ 103.842104] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.085Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1246gitlab # [ 103.851150] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.094Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1247gitlab # [ 103.858062] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.101Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1248gitlab # [ 103.865455] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.108Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1249gitlab # [ 103.871625] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.114Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1250gitlab # [ 103.877839] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.121Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1251gitlab # [ 103.882523] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.125Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1252gitlab # [ 103.890628] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.134Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1253gitlab # [ 103.905081] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.148Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1254gitlab # [ 103.909159] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.152Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1255gitlab # [ 103.913572] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.156Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1256gitlab # [ 103.918628] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.162Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1257gitlab # [ 103.923406] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.166Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1258gitlab # [ 103.928086] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.171Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1259gitlab # [ 103.932966] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.176Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1260gitlab # [ 103.937087] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.180Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1261gitlab # [ 103.942744] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.186Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1262gitlab # [ 103.948501] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.192Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1263gitlab # [ 103.953797] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:05.197Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1264gitlab # [ 108.327939] gitlab-workhorse[825]: time="2026-10-10T11:00:09Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1265gitlab # [ 118.336834] gitlab-workhorse[825]: time="2026-10-10T11:00:19Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1266gitlab # [ 128.160859] postgres[2063]: [2063] ERROR: relation "table_without_models" does not exist1267gitlab # [ 128.162191] postgres[2063]: [2063] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1268gitlab # [ 128.337745] gitlab-workhorse[825]: time="2026-10-10T11:00:29Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1269gitlab # [ 129.664728] postgres[2074]: [2074] ERROR: relation "table_without_models" does not exist1270gitlab # [ 129.665855] postgres[2074]: [2074] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1271gitlab # [ 130.721978] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:31.965Z","message":"Booted Rails 7.2.3.1 application in production environment"}1272gitlab # [ 130.724723] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:31.967Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1273gitlab # [ 130.725791] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:31.967Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1274gitlab # [ 130.726735] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:31.967Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1275gitlab # [ 130.729323] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:31.972Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1276gitlab # [ 132.072267] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:33.314Z","message":"Cleaning working queues"}1277gitlab # [ 132.077245] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:33.320Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"59ef4acebf8d3372a8c70f04","created_at":"2026-10-10T10:59:34.089Z","trace_propagation_headers":{"sentry-trace":"ce517c0e5e624406bf41ee39af41331d-c6c8445939b0423c","baggage":"sentry-trace_id=ce517c0e5e624406bf41ee39af41331d,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"51fc4f12704adc52d2bb6aad6bfa6b15","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-10T10:59:34.096Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-59ef4acebf8d3372a8c70f04: start","job_status":"start","queue_duration_s":59.22323,"scheduling_latency_s":59.223305}1278gitlab # [ 132.290691] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:00:33.533Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"59ef4acebf8d3372a8c70f04","created_at":"2026-10-10T10:59:34.089Z","trace_propagation_headers":{"sentry-trace":"ce517c0e5e624406bf41ee39af41331d-c6c8445939b0423c","baggage":"sentry-trace_id=ce517c0e5e624406bf41ee39af41331d,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"51fc4f12704adc52d2bb6aad6bfa6b15","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-10T10:59:34.096Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-59ef4acebf8d3372a8c70f04: done: 0.213953 sec","job_status":"done","queue_duration_s":59.22323,"scheduling_latency_s":59.223305,"redis_calls":28,"redis_duration_s":0.009054,"redis_read_bytes":408,"redis_write_bytes":3436,"redis_feature_flag_calls":17,"redis_feature_flag_duration_s":0.005921,"redis_feature_flag_read_bytes":398,"redis_feature_flag_write_bytes":2189,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001823,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00131,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":14,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":13,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076509,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.092442475,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.213953,"completed_at":"2026-10-10T11:00:33.533Z","load_balancing_strategy":"primary","db_duration_s":0.06297,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1279gitlab # [ 132.333430] bundle[1582]: {"timestamp":"2026-10-10T11:00:33.572Z","pid":1582,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1280gitlab # [ 132.335168] bundle[1582]: {"timestamp":"2026-10-10T11:00:33.572Z","pid":1582,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1281gitlab # [ 132.336689] bundle[1582]: {"timestamp":"2026-10-10T11:00:33.572Z","pid":1582,"message":"! #\u003cThread:0x00007601a8028738 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1282gitlab # [ 132.341421] bundle[1582]: {"timestamp":"2026-10-10T11:00:33.572Z","pid":1582,"message":"! #\u003cThread:0x00007601a82be538 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1283gitlab # [ 132.346200] bundle[1582]: {"timestamp":"2026-10-10T11:00:33.572Z","pid":1582,"message":"Use Ctrl-C to stop"}1284gitlab # [ 132.347420] bundle[1582]: {"timestamp":"2026-10-10T11:00:33.572Z","pid":1582,"message":"Sending status to systemd every 1.0 sec"}1285gitlab # [ 132.637725] bundle[1582]: {"timestamp":"2026-10-10T11:00:33.880Z","pid":1582,"message":"- Worker 0 (PID: 2175) booted in 0.28s, phase: 0"}1286gitlab # [ 132.665882] bundle[1582]: {"timestamp":"2026-10-10T11:00:33.909Z","pid":1582,"message":"- Worker 1 (PID: 2179) booted in 0.29s, phase: 0"}1287gitlab # [ 132.670378] systemd[1]: Started gitlab.service.1288gitlab # [ 132.672275] systemd[1]: Reached target Common target for all GitLab services..1289gitlab # [ 132.674687] systemd[1]: Starting setup-gitlab.service...1290gitlab # [ 132.705458] setup-gitlab-start[2203]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1291gitlab # [ 135.084426] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:00:36 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 23441292gitlab # [ 135.090089] setup-gitlab-start[2203]: + TOKEN=nixbot-test-token-of-20-chars1293gitlab # [ 135.092620] setup-gitlab-start[2203]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1294gitlab # [ 135.107296] runuser[2222]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1295gitlab # [ 135.489809] setup-gitlab-start[2223]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1296gitlab # [ 135.491537] setup-gitlab-start[2223]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1297gitlab # [ 135.493283] setup-gitlab-start[2223]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1298gitlab # [ 136.062376] setup-gitlab-start[2223]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1299gitlab # [ 136.063976] setup-gitlab-start[2223]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1300gitlab # [ 136.065506] setup-gitlab-start[2223]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1301gitlab # [ 160.636239] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:01.878Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"b3241b64e327b522cf33200d","created_at":"2026-10-10T11:01:01.783Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb9c1185722db4d17f750be63deba92c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:01.875Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-b3241b64e327b522cf33200d: start","job_status":"start","queue_duration_s":0.003173,"scheduling_latency_s":0.003215}1302gitlab # [ 160.742132] setup-gitlab-start[2223]: W, [2026-10-10T11:01:01.985035 #2223] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1303gitlab # [ 160.780047] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:02.023Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1304gitlab # [ 160.799893] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:02.043Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"b3241b64e327b522cf33200d","created_at":"2026-10-10T11:01:01.783Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fb9c1185722db4d17f750be63deba92c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:01.875Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-b3241b64e327b522cf33200d: done: 0.164448 sec","job_status":"done","queue_duration_s":0.003173,"scheduling_latency_s":0.003215,"redis_calls":10,"redis_duration_s":0.012945,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003404,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006994,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002547,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001432585,"duration_s":0.164448,"completed_at":"2026-10-10T11:01:02.043Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033206,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1305gitlab # [ 160.805298] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:02.048Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"cea4090569a0b0ffe0af3efa","created_at":"2026-10-10T11:01:01.910Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e8e00bc530c9c718d473d1efb605b22","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:01.928Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-cea4090569a0b0ffe0af3efa: start","job_status":"start","queue_duration_s":0.120201,"scheduling_latency_s":0.120242}1306gitlab # [ 160.939203] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:02.182Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"cea4090569a0b0ffe0af3efa","created_at":"2026-10-10T11:01:01.910Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e8e00bc530c9c718d473d1efb605b22","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:01.928Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-cea4090569a0b0ffe0af3efa: done: 0.133882 sec","job_status":"done","queue_duration_s":0.120201,"scheduling_latency_s":0.120242,"redis_calls":16,"redis_duration_s":0.027358,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.017832,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006759,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002767,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002400026,"duration_s":0.133882,"completed_at":"2026-10-10T11:01:02.182Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.053589,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1307gitlab # [ 160.981714] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:02.225Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"139db8f63100b4515d4c2ed8","created_at":"2026-10-10T11:01:01.939Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e07b232caf344a42b3f4a166a7817ff6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:01.957Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-139db8f63100b4515d4c2ed8: start","job_status":"start","queue_duration_s":0.267581,"scheduling_latency_s":0.267617}1308gitlab # [ 161.093286] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:02.336Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"139db8f63100b4515d4c2ed8","created_at":"2026-10-10T11:01:01.939Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e07b232caf344a42b3f4a166a7817ff6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:01.957Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-139db8f63100b4515d4c2ed8: done: 0.111142 sec","job_status":"done","queue_duration_s":0.267581,"scheduling_latency_s":0.267617,"redis_calls":12,"redis_duration_s":0.018045000000000002,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003171,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012467,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002407,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001898845,"duration_s":0.111142,"completed_at":"2026-10-10T11:01:02.336Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.036466,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 161.136575] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:02.379Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4877397a177494f124dfe2c8","created_at":"2026-10-10T11:01:01.972Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de99a773d893493e1098ebe8622915cc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:01.985Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4877397a177494f124dfe2c8: start","job_status":"start","queue_duration_s":0.394347,"scheduling_latency_s":0.394371}1310gitlab # [ 161.221127] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:02.464Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1311gitlab # [ 161.263562] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:02.506Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4877397a177494f124dfe2c8","created_at":"2026-10-10T11:01:01.972Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de99a773d893493e1098ebe8622915cc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:01.985Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4877397a177494f124dfe2c8: done: 0.12649 sec","job_status":"done","queue_duration_s":0.394347,"scheduling_latency_s":0.394371,"redis_calls":10,"redis_duration_s":0.019282,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003113,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.014101,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002068,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000951238,"duration_s":0.12649,"completed_at":"2026-10-10T11:01:02.506Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005049,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1312gitlab # [ 161.320531] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:02.563Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"7a795fd5dc38503844992b3c","created_at":"2026-10-10T11:01:01.994Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa86b3915788a1725b2dc16c004b15b8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.009Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-7a795fd5dc38503844992b3c: start","job_status":"start","queue_duration_s":0.553773,"scheduling_latency_s":0.553832}1313gitlab # [ 161.451173] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:02.694Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"7a795fd5dc38503844992b3c","created_at":"2026-10-10T11:01:01.994Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa86b3915788a1725b2dc16c004b15b8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.009Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-7a795fd5dc38503844992b3c: done: 0.130577 sec","job_status":"done","queue_duration_s":0.553773,"scheduling_latency_s":0.553832,"redis_calls":13,"redis_duration_s":0.024174,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003405,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.010557,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.010212,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.025939051,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.130577,"completed_at":"2026-10-10T11:01:02.694Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024774,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1314gitlab # [ 161.499200] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:02.742Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"94fb1cc7be6e1beee38997d3","created_at":"2026-10-10T11:01:02.018Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f78f920370eb87712f1248a1318c26c1","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.039Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-94fb1cc7be6e1beee38997d3: start","job_status":"start","queue_duration_s":0.702392,"scheduling_latency_s":0.70242}1315gitlab # [ 162.315454] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:03.558Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"94fb1cc7be6e1beee38997d3","created_at":"2026-10-10T11:01:02.018Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f78f920370eb87712f1248a1318c26c1","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.039Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-94fb1cc7be6e1beee38997d3: done: 0.815822 sec","job_status":"done","queue_duration_s":0.702392,"scheduling_latency_s":0.70242,"redis_calls":565,"redis_duration_s":0.499678,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002049,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004883,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.492746,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.315157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000774959,"duration_s":0.815822,"completed_at":"2026-10-10T11:01:03.558Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003681,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1316gitlab # [ 162.364305] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:03.607Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"6d18c5d1f9412f12df1fa2ca","created_at":"2026-10-10T11:01:02.047Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd7c5d4b4ddebe1cafd549723b159f9d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.062Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-6d18c5d1f9412f12df1fa2ca: start","job_status":"start","queue_duration_s":1.545624,"scheduling_latency_s":1.545653}1317gitlab # [ 162.424385] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:03.666Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"6d18c5d1f9412f12df1fa2ca","created_at":"2026-10-10T11:01:02.047Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd7c5d4b4ddebe1cafd549723b159f9d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.062Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-6d18c5d1f9412f12df1fa2ca: done: 0.058887 sec","job_status":"done","queue_duration_s":1.545624,"scheduling_latency_s":1.545653,"redis_calls":10,"redis_duration_s":0.015843,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003865,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009156,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002822,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001954997,"duration_s":0.058887,"completed_at":"2026-10-10T11:01:03.666Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00251,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 162.475566] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:03.718Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"889a877947b8add8fc655c2e","created_at":"2026-10-10T11:01:02.072Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"224204a8c7d0f768eba62ab2739c1ec5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.087Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-889a877947b8add8fc655c2e: start","job_status":"start","queue_duration_s":1.631009,"scheduling_latency_s":1.631061}1319gitlab # [ 162.504073] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:03.746Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1320gitlab # [ 162.533199] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:03.776Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"889a877947b8add8fc655c2e","created_at":"2026-10-10T11:01:02.072Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"224204a8c7d0f768eba62ab2739c1ec5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.087Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-889a877947b8add8fc655c2e: done: 0.057753 sec","job_status":"done","queue_duration_s":1.631009,"scheduling_latency_s":1.631061,"redis_calls":10,"redis_duration_s":0.012544,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002505,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008909,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00113,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016536,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00214301,"duration_s":0.057753,"completed_at":"2026-10-10T11:01:03.776Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00412,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1321gitlab # [ 162.555425] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:03.785Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"9ad0c6e9b1adf87c4a1e31ca","created_at":"2026-10-10T11:01:02.097Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"671ec4dcd1d4152cff12b8ac33643b0a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.110Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-9ad0c6e9b1adf87c4a1e31ca: start","job_status":"start","queue_duration_s":1.674438,"scheduling_latency_s":1.674474}1322gitlab # [ 162.620468] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:03.863Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"9ad0c6e9b1adf87c4a1e31ca","created_at":"2026-10-10T11:01:02.097Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"671ec4dcd1d4152cff12b8ac33643b0a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.110Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-9ad0c6e9b1adf87c4a1e31ca: done: 0.078141 sec","job_status":"done","queue_duration_s":1.674438,"scheduling_latency_s":1.674474,"redis_calls":10,"redis_duration_s":0.017218999999999998,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004176,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011292,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001751,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001722565,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.078141,"completed_at":"2026-10-10T11:01:03.863Z","load_balancing_strategy":"primary","db_duration_s":0.022576,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323gitlab # [ 162.641724] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:03.867Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"02dd194e9c92b112acc03bab","created_at":"2026-10-10T11:01:02.123Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.134Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-02dd194e9c92b112acc03bab: start","job_status":"start","queue_duration_s":1.732384,"scheduling_latency_s":1.732412}1324gitlab # [ 162.805794] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.049Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"02dd194e9c92b112acc03bab","created_at":"2026-10-10T11:01:02.123Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.134Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-02dd194e9c92b112acc03bab: done: 0.181952 sec","job_status":"done","queue_duration_s":1.732384,"scheduling_latency_s":1.732412,"redis_calls":71,"redis_duration_s":0.061996,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.040514,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001137,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.017879,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002466,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.099451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000295568,"duration_s":0.181952,"completed_at":"2026-10-10T11:01:04.048Z","load_balancing_strategy":"primary","db_duration_s":0.010655,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1325gitlab # [ 162.880228] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.050Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f02b99b182814570486a57a8","created_at":"2026-10-10T11:01:02.143Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd0846d9e0aae41c2e7d00325ebc5034","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.154Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f02b99b182814570486a57a8: start","job_status":"start","queue_duration_s":1.895974,"scheduling_latency_s":1.895989}1326gitlab # [ 163.043712] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.286Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f02b99b182814570486a57a8","created_at":"2026-10-10T11:01:02.143Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd0846d9e0aae41c2e7d00325ebc5034","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.154Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f02b99b182814570486a57a8: done: 0.236627 sec","job_status":"done","queue_duration_s":1.895974,"scheduling_latency_s":1.895989,"redis_calls":14,"redis_duration_s":0.003418,"redis_read_bytes":13,"redis_write_bytes":1641,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001061,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001224,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001133,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.084,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.043,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.126112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000464305,"exclusive_lock_hold_duration_s":0.177676772,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.236627,"completed_at":"2026-10-10T11:01:04.286Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.11968,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1327gitlab # [ 163.116381] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.288Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ab6ac7b2ae5c0b984edeac0e","created_at":"2026-10-10T11:01:02.169Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f19e252415eda22a7e9934fe9eabbc7f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.220Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ab6ac7b2ae5c0b984edeac0e: start","job_status":"start","queue_duration_s":2.067434,"scheduling_latency_s":2.067449}1328gitlab # [ 163.128744] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.321Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ab6ac7b2ae5c0b984edeac0e","created_at":"2026-10-10T11:01:02.169Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f19e252415eda22a7e9934fe9eabbc7f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.220Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ab6ac7b2ae5c0b984edeac0e: done: 0.033467 sec","job_status":"done","queue_duration_s":2.067434,"scheduling_latency_s":2.067449,"redis_calls":12,"redis_duration_s":0.002896,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000454,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001266,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001176,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000489447,"exclusive_lock_hold_duration_s":0.014713881,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.033467,"completed_at":"2026-10-10T11:01:04.321Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015728,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 163.166210] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.322Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f3f69efe3f884186b3a0d0f0","created_at":"2026-10-10T11:01:02.228Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28e1dcebced501109d5bd444dbb1fc47","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.241Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f3f69efe3f884186b3a0d0f0: start","job_status":"start","queue_duration_s":2.081542,"scheduling_latency_s":2.081557}1330gitlab # [ 163.178512] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.340Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f3f69efe3f884186b3a0d0f0","created_at":"2026-10-10T11:01:02.228Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28e1dcebced501109d5bd444dbb1fc47","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.241Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f3f69efe3f884186b3a0d0f0: done: 0.018022 sec","job_status":"done","queue_duration_s":2.081542,"scheduling_latency_s":2.081557,"redis_calls":10,"redis_duration_s":0.001446,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00034,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205613,"duration_s":0.018022,"completed_at":"2026-10-10T11:01:04.340Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003935,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331gitlab # [ 163.179993] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.341Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f4724048a8e19f9e77532d78","created_at":"2026-10-10T11:01:02.252Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57b6919d5dbe3b48b90b4cd82823e31b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.261Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f4724048a8e19f9e77532d78: start","job_status":"start","queue_duration_s":2.080558,"scheduling_latency_s":2.080571}1332gitlab # [ 163.189510] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.350Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1333gitlab # [ 163.192087] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.354Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f4724048a8e19f9e77532d78","created_at":"2026-10-10T11:01:02.252Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57b6919d5dbe3b48b90b4cd82823e31b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.261Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f4724048a8e19f9e77532d78: done: 0.012115 sec","job_status":"done","queue_duration_s":2.080558,"scheduling_latency_s":2.080571,"redis_calls":10,"redis_duration_s":0.001351,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000283,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00081,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000258,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000199746,"duration_s":0.012115,"completed_at":"2026-10-10T11:01:04.354Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000483,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 163.238520] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.354Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"f821486908f4d93cf61b7c6b","created_at":"2026-10-10T11:01:02.290Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cebb852efb5b9c01e888753740ae0b1","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.307Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-f821486908f4d93cf61b7c6b: start","job_status":"start","queue_duration_s":2.047905,"scheduling_latency_s":2.047918}1335gitlab # [ 163.240024] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.372Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"f821486908f4d93cf61b7c6b","created_at":"2026-10-10T11:01:02.290Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cebb852efb5b9c01e888753740ae0b1","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.307Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-f821486908f4d93cf61b7c6b: done: 0.01751 sec","job_status":"done","queue_duration_s":2.047905,"scheduling_latency_s":2.047918,"redis_calls":12,"redis_duration_s":0.00211,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000334,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000816,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000332165,"exclusive_lock_hold_duration_s":0.0031104509999999998,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.01751,"completed_at":"2026-10-10T11:01:04.372Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002505,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336gitlab # [ 163.294671] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.373Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"05b83adf5be6fbf1c82866be","created_at":"2026-10-10T11:01:02.326Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2df3f717c433da4e74110fbd6976e25","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.378Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-05b83adf5be6fbf1c82866be: start","job_status":"start","queue_duration_s":1.994739,"scheduling_latency_s":1.994755}1337gitlab # [ 163.295459] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.396Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"05b83adf5be6fbf1c82866be","created_at":"2026-10-10T11:01:02.326Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d2df3f717c433da4e74110fbd6976e25","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.378Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-05b83adf5be6fbf1c82866be: done: 0.022937 sec","job_status":"done","queue_duration_s":1.994739,"scheduling_latency_s":1.994755,"redis_calls":19,"redis_duration_s":0.003117,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.0007,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000243,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001658,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000516,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000244724,"duration_s":0.022937,"completed_at":"2026-10-10T11:01:04.396Z","load_balancing_strategy":"primary","db_duration_s":0.001654,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338gitlab # [ 163.325592] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.397Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"73ac57dadbb5bb7d58541d81","created_at":"2026-10-10T11:01:02.407Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da735a9ee4aa807ec89230b11bd8c07a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.431Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-73ac57dadbb5bb7d58541d81: start","job_status":"start","queue_duration_s":1.96573,"scheduling_latency_s":1.965744}1339gitlab # [ 163.366124] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.413Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"73ac57dadbb5bb7d58541d81","created_at":"2026-10-10T11:01:02.407Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da735a9ee4aa807ec89230b11bd8c07a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.431Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-73ac57dadbb5bb7d58541d81: done: 0.016261 sec","job_status":"done","queue_duration_s":1.96573,"scheduling_latency_s":1.965744,"redis_calls":12,"redis_duration_s":0.002133,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000336,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000845,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000368761,"exclusive_lock_hold_duration_s":0.004757029,"duration_s":0.016261,"completed_at":"2026-10-10T11:01:04.413Z","load_balancing_strategy":"primary","db_duration_s":0.002583,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 163.371609] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.414Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"86769c5a58d2dddee6eefc4b","created_at":"2026-10-10T11:01:02.443Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85ab86fac222e28f12e82d6a46b86fd8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-10T11:01:02.462Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-86769c5a58d2dddee6eefc4b: start","job_status":"start","queue_duration_s":1.952454,"scheduling_latency_s":1.952468}1341gitlab # [ 163.418297] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.422Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1342gitlab # [ 163.418980] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.426Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"86769c5a58d2dddee6eefc4b","created_at":"2026-10-10T11:01:02.443Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85ab86fac222e28f12e82d6a46b86fd8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-10T11:01:02.462Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-86769c5a58d2dddee6eefc4b: done: 0.011449 sec","job_status":"done","queue_duration_s":1.952454,"scheduling_latency_s":1.952468,"redis_calls":10,"redis_duration_s":0.001618,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000399,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000324,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000238857,"duration_s":0.011449,"completed_at":"2026-10-10T11:01:04.426Z","load_balancing_strategy":"primary","db_duration_s":0.000386,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1343gitlab # [ 163.420941] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.427Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"5a312e27394850be68c79505","created_at":"2026-10-10T11:01:02.488Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46dddcb26a8cea0e30298ba4031c9ac7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-10T11:01:02.559Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-5a312e27394850be68c79505: start","job_status":"start","queue_duration_s":1.867728,"scheduling_latency_s":1.86774}1344gitlab # [ 163.465141] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.434Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1345gitlab # [ 163.465759] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.438Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"5a312e27394850be68c79505","created_at":"2026-10-10T11:01:02.488Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46dddcb26a8cea0e30298ba4031c9ac7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-10T11:01:02.559Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-5a312e27394850be68c79505: done: 0.011328 sec","job_status":"done","queue_duration_s":1.867728,"scheduling_latency_s":1.86774,"redis_calls":10,"redis_duration_s":0.00125,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000234,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000763,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000253,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000226286,"duration_s":0.011328,"completed_at":"2026-10-10T11:01:04.438Z","load_balancing_strategy":"primary","db_duration_s":0.001142,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 163.469591] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.439Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"969ba105222414c503536d08","created_at":"2026-10-10T11:01:02.596Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c6877f49e474428dc8d79abd71f75d0b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.607Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-969ba105222414c503536d08: start","job_status":"start","queue_duration_s":1.83206,"scheduling_latency_s":1.832073}1347gitlab # [ 163.482166] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.461Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"969ba105222414c503536d08","created_at":"2026-10-10T11:01:02.596Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c6877f49e474428dc8d79abd71f75d0b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:02.607Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-969ba105222414c503536d08: done: 0.022496 sec","job_status":"done","queue_duration_s":1.83206,"scheduling_latency_s":1.832073,"redis_calls":12,"redis_duration_s":0.0015689999999999999,"redis_read_bytes":11,"redis_write_bytes":1612,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0003,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000694,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000575,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000320711,"exclusive_lock_hold_duration_s":0.009962719,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.022496,"completed_at":"2026-10-10T11:01:04.461Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005572,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348gitlab # [ 163.523139] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.462Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a9cf45d58023a482e295fa31","created_at":"2026-10-10T11:01:02.625Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d6f6fa0177b4018c7a55d221b366b83","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-10T11:01:02.640Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a9cf45d58023a482e295fa31: start","job_status":"start","queue_duration_s":1.82273,"scheduling_latency_s":1.822743}1349gitlab # [ 163.530614] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.480Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a9cf45d58023a482e295fa31","created_at":"2026-10-10T11:01:02.625Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6235d1413a024969","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d6f6fa0177b4018c7a55d221b366b83","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-10T11:01:02.640Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a9cf45d58023a482e295fa31: done: 0.017967 sec","job_status":"done","queue_duration_s":1.82273,"scheduling_latency_s":1.822743,"redis_calls":10,"redis_duration_s":0.00125,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000233,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000226844,"duration_s":0.017967,"completed_at":"2026-10-10T11:01:04.480Z","load_balancing_strategy":"primary","db_duration_s":0.002241,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 163.567244] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.481Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42aa346936356ff285f9e9f4","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42aa346936356ff285f9e9f4: start","job_status":"start","queue_duration_s":0.440183,"scheduling_latency_s":0.440197}1351gitlab # [ 163.574930] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.496Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42aa346936356ff285f9e9f4","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42aa346936356ff285f9e9f4: done: 0.015095 sec","job_status":"done","queue_duration_s":0.440183,"scheduling_latency_s":0.440197,"redis_calls":16,"redis_duration_s":0.002003,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000635,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001126,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000197232,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015095,"completed_at":"2026-10-10T11:01:04.496Z","load_balancing_strategy":"primary","db_duration_s":0.001154,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1352gitlab # [ 163.619109] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af4ccd8d2001cdc40c7111fd","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af4ccd8d2001cdc40c7111fd: start","job_status":"start","queue_duration_s":0.456157,"scheduling_latency_s":0.45617}1353gitlab # [ 163.619750] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af4ccd8d2001cdc40c7111fd","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af4ccd8d2001cdc40c7111fd: done: 0.013449 sec","job_status":"done","queue_duration_s":0.456157,"scheduling_latency_s":0.45617,"redis_calls":13,"redis_duration_s":0.001822,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000613,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001209,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013449,"completed_at":"2026-10-10T11:01:04.511Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354gitlab # [ 163.659217] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.512Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"230cad3a307331d534613aec","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-230cad3a307331d534613aec: start","job_status":"start","queue_duration_s":0.470641,"scheduling_latency_s":0.470664}1355gitlab # [ 163.662541] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"230cad3a307331d534613aec","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-230cad3a307331d534613aec: done: 0.010671 sec","job_status":"done","queue_duration_s":0.470641,"scheduling_latency_s":0.470664,"redis_calls":13,"redis_duration_s":0.001527,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000451,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010671,"completed_at":"2026-10-10T11:01:04.522Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1356gitlab # [ 163.721989] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf903b6bdf57eb3822734bbe","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf903b6bdf57eb3822734bbe: start","job_status":"start","queue_duration_s":0.482134,"scheduling_latency_s":0.482147}1357gitlab # [ 163.722768] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.533Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf903b6bdf57eb3822734bbe","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf903b6bdf57eb3822734bbe: done: 0.009671 sec","job_status":"done","queue_duration_s":0.482134,"scheduling_latency_s":0.482147,"redis_calls":13,"redis_duration_s":0.001818,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009671,"completed_at":"2026-10-10T11:01:04.533Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 163.741233] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a62b6c5d388a1db4ddd256f2","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a62b6c5d388a1db4ddd256f2: start","job_status":"start","queue_duration_s":0.492855,"scheduling_latency_s":0.492869}1359gitlab # [ 163.751950] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a62b6c5d388a1db4ddd256f2","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a62b6c5d388a1db4ddd256f2: done: 0.011588 sec","job_status":"done","queue_duration_s":0.492855,"scheduling_latency_s":0.492869,"redis_calls":13,"redis_duration_s":0.001682,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001129,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011588,"completed_at":"2026-10-10T11:01:04.546Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 163.805976] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.547Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b394b596afb7f33efc0e7d0b","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b394b596afb7f33efc0e7d0b: start","job_status":"start","queue_duration_s":0.505693,"scheduling_latency_s":0.505711}1361gitlab # [ 163.806754] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.557Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b394b596afb7f33efc0e7d0b","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b394b596afb7f33efc0e7d0b: done: 0.010166 sec","job_status":"done","queue_duration_s":0.505693,"scheduling_latency_s":0.505711,"redis_calls":13,"redis_duration_s":0.001494,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006694,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010166,"completed_at":"2026-10-10T11:01:04.557Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 163.859207] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1df24036a572a2f4e9e0a984","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1df24036a572a2f4e9e0a984: start","job_status":"start","queue_duration_s":0.516985,"scheduling_latency_s":0.516999}1363gitlab # [ 163.859873] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1df24036a572a2f4e9e0a984","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1df24036a572a2f4e9e0a984: done: 0.009975 sec","job_status":"done","queue_duration_s":0.516985,"scheduling_latency_s":0.516999,"redis_calls":13,"redis_duration_s":0.001542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009975,"completed_at":"2026-10-10T11:01:04.568Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 163.876876] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"255b66ff436a144f45677182","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-255b66ff436a144f45677182: start","job_status":"start","queue_duration_s":0.528091,"scheduling_latency_s":0.528129}1365gitlab # [ 163.915223] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.579Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"255b66ff436a144f45677182","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-255b66ff436a144f45677182: done: 0.009937 sec","job_status":"done","queue_duration_s":0.528091,"scheduling_latency_s":0.528129,"redis_calls":13,"redis_duration_s":0.001633,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009937,"completed_at":"2026-10-10T11:01:04.579Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 163.957658] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.580Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"919107d26b69edb185c36af8","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-919107d26b69edb185c36af8: start","job_status":"start","queue_duration_s":0.539084,"scheduling_latency_s":0.539123}1367gitlab # [ 163.958515] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.590Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"919107d26b69edb185c36af8","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-919107d26b69edb185c36af8: done: 0.009578 sec","job_status":"done","queue_duration_s":0.539084,"scheduling_latency_s":0.539123,"redis_calls":13,"redis_duration_s":0.001424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009578,"completed_at":"2026-10-10T11:01:04.590Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 163.966604] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f45a13ef5227edd17e2c751","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f45a13ef5227edd17e2c751: start","job_status":"start","queue_duration_s":0.549461,"scheduling_latency_s":0.549473}1369gitlab # [ 163.977684] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f45a13ef5227edd17e2c751","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f45a13ef5227edd17e2c751: done: 0.009769 sec","job_status":"done","queue_duration_s":0.549461,"scheduling_latency_s":0.549473,"redis_calls":13,"redis_duration_s":0.001706,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001219,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006483,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009769,"completed_at":"2026-10-10T11:01:04.600Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 164.028735] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"732468c3b9ca8f657926bbe3","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-732468c3b9ca8f657926bbe3: start","job_status":"start","queue_duration_s":0.56001,"scheduling_latency_s":0.560022}1371gitlab # [ 164.029040] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"732468c3b9ca8f657926bbe3","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-732468c3b9ca8f657926bbe3: done: 0.010452 sec","job_status":"done","queue_duration_s":0.56001,"scheduling_latency_s":0.560022,"redis_calls":13,"redis_duration_s":0.00181,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000468,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001342,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010452,"completed_at":"2026-10-10T11:01:04.612Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 164.072917] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a442ffe1f9b8027f79f550fd","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a442ffe1f9b8027f79f550fd: start","job_status":"start","queue_duration_s":0.571302,"scheduling_latency_s":0.571314}1373gitlab # [ 164.073648] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.623Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a442ffe1f9b8027f79f550fd","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a442ffe1f9b8027f79f550fd: done: 0.010273 sec","job_status":"done","queue_duration_s":0.571302,"scheduling_latency_s":0.571314,"redis_calls":13,"redis_duration_s":0.001717,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010273,"completed_at":"2026-10-10T11:01:04.623Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 164.135686] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2a9b519a220c10fd677fea7","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2a9b519a220c10fd677fea7: start","job_status":"start","queue_duration_s":0.582393,"scheduling_latency_s":0.582407}1375gitlab # [ 164.135930] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2a9b519a220c10fd677fea7","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2a9b519a220c10fd677fea7: done: 0.013415 sec","job_status":"done","queue_duration_s":0.582393,"scheduling_latency_s":0.582407,"redis_calls":13,"redis_duration_s":0.002198,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00165,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013415,"completed_at":"2026-10-10T11:01:04.637Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 164.147396] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.638Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09a93dd265abb4a96b8fceaf","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09a93dd265abb4a96b8fceaf: start","job_status":"start","queue_duration_s":0.596872,"scheduling_latency_s":0.596894}1377gitlab # [ 164.186733] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09a93dd265abb4a96b8fceaf","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09a93dd265abb4a96b8fceaf: done: 0.009945 sec","job_status":"done","queue_duration_s":0.596872,"scheduling_latency_s":0.596894,"redis_calls":13,"redis_duration_s":0.001436,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000456,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009945,"completed_at":"2026-10-10T11:01:04.648Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 164.192617] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e2495920f10a6dbde78705a","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e2495920f10a6dbde78705a: start","job_status":"start","queue_duration_s":0.607666,"scheduling_latency_s":0.607681}1379gitlab # [ 164.234146] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e2495920f10a6dbde78705a","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e2495920f10a6dbde78705a: done: 0.008762 sec","job_status":"done","queue_duration_s":0.607666,"scheduling_latency_s":0.607681,"redis_calls":13,"redis_duration_s":0.001429,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005825,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008762,"completed_at":"2026-10-10T11:01:04.657Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 164.274309] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"764a29a5202c71f4cf9e611e","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-764a29a5202c71f4cf9e611e: start","job_status":"start","queue_duration_s":0.617235,"scheduling_latency_s":0.617248}1381gitlab # [ 164.275055] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"764a29a5202c71f4cf9e611e","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-764a29a5202c71f4cf9e611e: done: 0.010258 sec","job_status":"done","queue_duration_s":0.617235,"scheduling_latency_s":0.617248,"redis_calls":13,"redis_duration_s":0.00163,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010258,"completed_at":"2026-10-10T11:01:04.669Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 164.312882] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.670Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3d246aeddae0ec0c2aef155","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3d246aeddae0ec0c2aef155: start","job_status":"start","queue_duration_s":0.628439,"scheduling_latency_s":0.628456}1383gitlab # [ 164.313674] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3d246aeddae0ec0c2aef155","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3d246aeddae0ec0c2aef155: done: 0.009164 sec","job_status":"done","queue_duration_s":0.628439,"scheduling_latency_s":0.628456,"redis_calls":13,"redis_duration_s":0.001551,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009164,"completed_at":"2026-10-10T11:01:04.679Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 164.327971] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d18e09021a579b64351da3af","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d18e09021a579b64351da3af: start","job_status":"start","queue_duration_s":0.638403,"scheduling_latency_s":0.638415}1385gitlab # [ 164.338567] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d18e09021a579b64351da3af","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d18e09021a579b64351da3af: done: 0.008621 sec","job_status":"done","queue_duration_s":0.638403,"scheduling_latency_s":0.638415,"redis_calls":13,"redis_duration_s":0.00142,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000468,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008621,"completed_at":"2026-10-10T11:01:04.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 164.393624] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa80583574b6c3a5a388f56c","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa80583574b6c3a5a388f56c: start","job_status":"start","queue_duration_s":0.647815,"scheduling_latency_s":0.647829}1387gitlab # [ 164.394546] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa80583574b6c3a5a388f56c","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa80583574b6c3a5a388f56c: done: 0.008583 sec","job_status":"done","queue_duration_s":0.647815,"scheduling_latency_s":0.647829,"redis_calls":13,"redis_duration_s":0.001402,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008583,"completed_at":"2026-10-10T11:01:04.697Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 164.418583] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.698Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"725d7eec341e44c3a6aac6bd","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-725d7eec341e44c3a6aac6bd: start","job_status":"start","queue_duration_s":0.657199,"scheduling_latency_s":0.657212}1389gitlab # [ 164.429720] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"725d7eec341e44c3a6aac6bd","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-725d7eec341e44c3a6aac6bd: done: 0.008736 sec","job_status":"done","queue_duration_s":0.657199,"scheduling_latency_s":0.657212,"redis_calls":13,"redis_duration_s":0.001483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000463,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008736,"completed_at":"2026-10-10T11:01:04.707Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 164.474163] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cbc21de3e6ffa2c0ebc751b","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cbc21de3e6ffa2c0ebc751b: start","job_status":"start","queue_duration_s":0.666757,"scheduling_latency_s":0.666771}1391gitlab # [ 164.474797] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.717Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cbc21de3e6ffa2c0ebc751b","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cbc21de3e6ffa2c0ebc751b: done: 0.008871 sec","job_status":"done","queue_duration_s":0.666757,"scheduling_latency_s":0.666771,"redis_calls":13,"redis_duration_s":0.00154,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005547,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008871,"completed_at":"2026-10-10T11:01:04.717Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392gitlab # [ 164.513777] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.718Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5db7584853b80b5df7cce4db","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5db7584853b80b5df7cce4db: start","job_status":"start","queue_duration_s":0.676509,"scheduling_latency_s":0.676522}1393gitlab # [ 164.552789] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5db7584853b80b5df7cce4db","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5db7584853b80b5df7cce4db: done: 0.00905 sec","job_status":"done","queue_duration_s":0.676509,"scheduling_latency_s":0.676522,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00905,"completed_at":"2026-10-10T11:01:04.727Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 164.553839] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7eacd4910430345c6f75b96","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7eacd4910430345c6f75b96: start","job_status":"start","queue_duration_s":0.686382,"scheduling_latency_s":0.686396}1395gitlab # [ 164.565019] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7eacd4910430345c6f75b96","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7eacd4910430345c6f75b96: done: 0.008724 sec","job_status":"done","queue_duration_s":0.686382,"scheduling_latency_s":0.686396,"redis_calls":13,"redis_duration_s":0.001506,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008724,"completed_at":"2026-10-10T11:01:04.736Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 164.634350] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"552e69810579338943f03ca2","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-552e69810579338943f03ca2: start","job_status":"start","queue_duration_s":0.695908,"scheduling_latency_s":0.695921}1397gitlab # [ 164.635097] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"552e69810579338943f03ca2","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-552e69810579338943f03ca2: done: 0.008693 sec","job_status":"done","queue_duration_s":0.695908,"scheduling_latency_s":0.695921,"redis_calls":13,"redis_duration_s":0.001398,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000453,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008693,"completed_at":"2026-10-10T11:01:04.746Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 164.679186] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.747Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8fe16ccdc248b649841cc57","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8fe16ccdc248b649841cc57: start","job_status":"start","queue_duration_s":0.705455,"scheduling_latency_s":0.705467}1399gitlab # [ 164.679904] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8fe16ccdc248b649841cc57","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8fe16ccdc248b649841cc57: done: 0.008705 sec","job_status":"done","queue_duration_s":0.705455,"scheduling_latency_s":0.705467,"redis_calls":13,"redis_duration_s":0.001496,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008705,"completed_at":"2026-10-10T11:01:04.755Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 164.690720] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5476d75884d20851952166d5","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5476d75884d20851952166d5: start","job_status":"start","queue_duration_s":0.714923,"scheduling_latency_s":0.714937}1401gitlab # [ 164.734635] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5476d75884d20851952166d5","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5476d75884d20851952166d5: done: 0.008704 sec","job_status":"done","queue_duration_s":0.714923,"scheduling_latency_s":0.714937,"redis_calls":13,"redis_duration_s":0.0014919999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000468,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008704,"completed_at":"2026-10-10T11:01:04.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 164.735960] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c033d7fe1cc3f1629f00e17","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c033d7fe1cc3f1629f00e17: start","job_status":"start","queue_duration_s":0.724419,"scheduling_latency_s":0.724432}1403gitlab # [ 164.747059] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c033d7fe1cc3f1629f00e17","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c033d7fe1cc3f1629f00e17: done: 0.008841 sec","job_status":"done","queue_duration_s":0.724419,"scheduling_latency_s":0.724432,"redis_calls":13,"redis_duration_s":0.001546,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008841,"completed_at":"2026-10-10T11:01:04.774Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 164.794512] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9620708a37db2067af6fd534","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9620708a37db2067af6fd534: start","job_status":"start","queue_duration_s":0.734056,"scheduling_latency_s":0.734068}1405gitlab # [ 164.795251] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9620708a37db2067af6fd534","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9620708a37db2067af6fd534: done: 0.008682 sec","job_status":"done","queue_duration_s":0.734056,"scheduling_latency_s":0.734068,"redis_calls":13,"redis_duration_s":0.001423,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000459,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008682,"completed_at":"2026-10-10T11:01:04.784Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 164.844109] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"caf3739a124b93c96e36106e","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-caf3739a124b93c96e36106e: start","job_status":"start","queue_duration_s":0.743533,"scheduling_latency_s":0.743546}1407gitlab # [ 164.844726] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.793Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"caf3739a124b93c96e36106e","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-caf3739a124b93c96e36106e: done: 0.008544 sec","job_status":"done","queue_duration_s":0.743533,"scheduling_latency_s":0.743546,"redis_calls":13,"redis_duration_s":0.001391,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000457,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008544,"completed_at":"2026-10-10T11:01:04.793Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 164.898666] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.794Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"163b741a2811f2631e019746","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-163b741a2811f2631e019746: start","job_status":"start","queue_duration_s":0.752908,"scheduling_latency_s":0.752921}1409gitlab # [ 164.899585] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.803Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"163b741a2811f2631e019746","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-163b741a2811f2631e019746: done: 0.008576 sec","job_status":"done","queue_duration_s":0.752908,"scheduling_latency_s":0.752921,"redis_calls":13,"redis_duration_s":0.001394,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000449,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008576,"completed_at":"2026-10-10T11:01:04.803Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 164.953707] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.803Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71f5b85ee0e21e8050dd6a5f","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71f5b85ee0e21e8050dd6a5f: start","job_status":"start","queue_duration_s":0.762281,"scheduling_latency_s":0.762294}1411gitlab # [ 164.954637] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71f5b85ee0e21e8050dd6a5f","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71f5b85ee0e21e8050dd6a5f: done: 0.008813 sec","job_status":"done","queue_duration_s":0.762281,"scheduling_latency_s":0.762294,"redis_calls":13,"redis_duration_s":0.001536,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001048,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008813,"completed_at":"2026-10-10T11:01:04.812Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 164.962648] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.813Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec6a3822fce7997fdf9b890b","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec6a3822fce7997fdf9b890b: start","job_status":"start","queue_duration_s":0.771912,"scheduling_latency_s":0.771925}1413gitlab # [ 165.001593] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec6a3822fce7997fdf9b890b","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec6a3822fce7997fdf9b890b: done: 0.009061 sec","job_status":"done","queue_duration_s":0.771912,"scheduling_latency_s":0.771925,"redis_calls":13,"redis_duration_s":0.001687,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000483,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001204,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00565,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009061,"completed_at":"2026-10-10T11:01:04.822Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414gitlab # [ 165.040706] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6e0e1b87859003472dc8bd2","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6e0e1b87859003472dc8bd2: start","job_status":"start","queue_duration_s":0.78179,"scheduling_latency_s":0.781803}1415gitlab # [ 165.041501] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6e0e1b87859003472dc8bd2","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6e0e1b87859003472dc8bd2: done: 0.008728 sec","job_status":"done","queue_duration_s":0.78179,"scheduling_latency_s":0.781803,"redis_calls":13,"redis_duration_s":0.0015179999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005684,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008728,"completed_at":"2026-10-10T11:01:04.832Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 165.080793] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b7182f0b8433a305fb911b1","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b7182f0b8433a305fb911b1: start","job_status":"start","queue_duration_s":0.791357,"scheduling_latency_s":0.79137}1417gitlab # [ 165.081763] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.841Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b7182f0b8433a305fb911b1","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b7182f0b8433a305fb911b1: done: 0.008608 sec","job_status":"done","queue_duration_s":0.791357,"scheduling_latency_s":0.79137,"redis_calls":13,"redis_duration_s":0.0013800000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000456,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005735,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008608,"completed_at":"2026-10-10T11:01:04.841Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418gitlab # [ 165.099151] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9dc505b1539367890b550eea","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9dc505b1539367890b550eea: start","job_status":"start","queue_duration_s":0.800753,"scheduling_latency_s":0.800766}1419gitlab # [ 165.139807] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9dc505b1539367890b550eea","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9dc505b1539367890b550eea: done: 0.008598 sec","job_status":"done","queue_duration_s":0.800753,"scheduling_latency_s":0.800766,"redis_calls":13,"redis_duration_s":0.001439,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008598,"completed_at":"2026-10-10T11:01:04.850Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420gitlab # [ 165.178769] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.851Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e202b48ce63e684cb9e33e5","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e202b48ce63e684cb9e33e5: start","job_status":"start","queue_duration_s":0.810133,"scheduling_latency_s":0.810145}1421gitlab # [ 165.179718] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.860Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e202b48ce63e684cb9e33e5","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e202b48ce63e684cb9e33e5: done: 0.008687 sec","job_status":"done","queue_duration_s":0.810133,"scheduling_latency_s":0.810145,"redis_calls":13,"redis_duration_s":0.00142,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000459,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005794,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008687,"completed_at":"2026-10-10T11:01:04.860Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 165.189829] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb68b8c4baa481a445420985","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb68b8c4baa481a445420985: start","job_status":"start","queue_duration_s":0.819588,"scheduling_latency_s":0.819602}1423gitlab # [ 165.201681] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.870Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb68b8c4baa481a445420985","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb68b8c4baa481a445420985: done: 0.008862 sec","job_status":"done","queue_duration_s":0.819588,"scheduling_latency_s":0.819602,"redis_calls":13,"redis_duration_s":0.0016,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008862,"completed_at":"2026-10-10T11:01:04.870Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 165.237447] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.870Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3d7604a4cf8ca5c8429d6a8","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3d7604a4cf8ca5c8429d6a8: start","job_status":"start","queue_duration_s":0.829249,"scheduling_latency_s":0.829264}1425gitlab # [ 165.276203] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3d7604a4cf8ca5c8429d6a8","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3d7604a4cf8ca5c8429d6a8: done: 0.008561 sec","job_status":"done","queue_duration_s":0.829249,"scheduling_latency_s":0.829264,"redis_calls":13,"redis_duration_s":0.001457,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000451,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008561,"completed_at":"2026-10-10T11:01:04.879Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426gitlab # [ 165.282646] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ebc71b5886b808402084d5f","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ebc71b5886b808402084d5f: start","job_status":"start","queue_duration_s":0.838603,"scheduling_latency_s":0.838617}1427gitlab # [ 165.328108] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ebc71b5886b808402084d5f","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ebc71b5886b808402084d5f: done: 0.008607 sec","job_status":"done","queue_duration_s":0.838603,"scheduling_latency_s":0.838617,"redis_calls":13,"redis_duration_s":0.0014269999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000486,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008607,"completed_at":"2026-10-10T11:01:04.888Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428gitlab # [ 165.329112] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ac5de939b40caa8a712cb26","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ac5de939b40caa8a712cb26: start","job_status":"start","queue_duration_s":0.847991,"scheduling_latency_s":0.848004}1429gitlab # [ 165.339102] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ac5de939b40caa8a712cb26","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ac5de939b40caa8a712cb26: done: 0.008596 sec","job_status":"done","queue_duration_s":0.847991,"scheduling_latency_s":0.848004,"redis_calls":13,"redis_duration_s":0.001393,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000453,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008596,"completed_at":"2026-10-10T11:01:04.898Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430gitlab # [ 165.374119] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae90b2587cb345bd40b00287","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae90b2587cb345bd40b00287: start","job_status":"start","queue_duration_s":0.85747,"scheduling_latency_s":0.857483}1431gitlab # [ 165.385212] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae90b2587cb345bd40b00287","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae90b2587cb345bd40b00287: done: 0.008794 sec","job_status":"done","queue_duration_s":0.85747,"scheduling_latency_s":0.857483,"redis_calls":13,"redis_duration_s":0.001565,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008794,"completed_at":"2026-10-10T11:01:04.907Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 165.438273] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"383fdc560188945a00a98698","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-383fdc560188945a00a98698: start","job_status":"start","queue_duration_s":0.867167,"scheduling_latency_s":0.86718}1433gitlab # [ 165.439064] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"383fdc560188945a00a98698","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-383fdc560188945a00a98698: done: 0.008745 sec","job_status":"done","queue_duration_s":0.867167,"scheduling_latency_s":0.86718,"redis_calls":13,"redis_duration_s":0.001483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000472,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008745,"completed_at":"2026-10-10T11:01:04.917Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434gitlab # [ 165.489075] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11cc65ca38ede3ba0264a7ed","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11cc65ca38ede3ba0264a7ed: start","job_status":"start","queue_duration_s":0.876741,"scheduling_latency_s":0.876754}1435gitlab # [ 165.489715] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.927Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11cc65ca38ede3ba0264a7ed","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11cc65ca38ede3ba0264a7ed: done: 0.009046 sec","job_status":"done","queue_duration_s":0.876741,"scheduling_latency_s":0.876754,"redis_calls":13,"redis_duration_s":0.0016640000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000486,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009046,"completed_at":"2026-10-10T11:01:04.927Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436gitlab # [ 165.538145] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.928Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dea6482d46d8be8abebf74b0","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dea6482d46d8be8abebf74b0: start","job_status":"start","queue_duration_s":0.886615,"scheduling_latency_s":0.886628}1437gitlab # [ 165.538872] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dea6482d46d8be8abebf74b0","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dea6482d46d8be8abebf74b0: done: 0.008693 sec","job_status":"done","queue_duration_s":0.886615,"scheduling_latency_s":0.886628,"redis_calls":13,"redis_duration_s":0.001473,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000475,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005684,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008693,"completed_at":"2026-10-10T11:01:04.936Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1438gitlab # [ 165.581513] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5789baed736ca631efec975","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5789baed736ca631efec975: start","job_status":"start","queue_duration_s":0.896087,"scheduling_latency_s":0.896124}1439gitlab # [ 165.582130] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5789baed736ca631efec975","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5789baed736ca631efec975: done: 0.008638 sec","job_status":"done","queue_duration_s":0.896087,"scheduling_latency_s":0.896124,"redis_calls":13,"redis_duration_s":0.001443,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000458,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008638,"completed_at":"2026-10-10T11:01:04.946Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440gitlab # [ 165.635523] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cc3801ddbeac8ffa748d96f","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cc3801ddbeac8ffa748d96f: start","job_status":"start","queue_duration_s":0.905546,"scheduling_latency_s":0.905558}1441gitlab # [ 165.636436] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cc3801ddbeac8ffa748d96f","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cc3801ddbeac8ffa748d96f: done: 0.00863 sec","job_status":"done","queue_duration_s":0.905546,"scheduling_latency_s":0.905558,"redis_calls":13,"redis_duration_s":0.00141,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000473,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00863,"completed_at":"2026-10-10T11:01:04.955Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442gitlab # [ 165.676737] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b144532fb412acbbd403e970","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b144532fb412acbbd403e970: start","job_status":"start","queue_duration_s":0.914941,"scheduling_latency_s":0.914954}1443gitlab # [ 165.677638] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b144532fb412acbbd403e970","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b144532fb412acbbd403e970: done: 0.008637 sec","job_status":"done","queue_duration_s":0.914941,"scheduling_latency_s":0.914954,"redis_calls":13,"redis_duration_s":0.001388,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008637,"completed_at":"2026-10-10T11:01:04.965Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444gitlab # [ 165.690710] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5022bae4a3c70c1152a0f6e9","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5022bae4a3c70c1152a0f6e9: start","job_status":"start","queue_duration_s":0.924461,"scheduling_latency_s":0.924474}1445gitlab # [ 165.733380] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5022bae4a3c70c1152a0f6e9","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5022bae4a3c70c1152a0f6e9: done: 0.008862 sec","job_status":"done","queue_duration_s":0.924461,"scheduling_latency_s":0.924474,"redis_calls":13,"redis_duration_s":0.001567,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008862,"completed_at":"2026-10-10T11:01:04.974Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 165.772620] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8c90d8c3233ebbb75ee830d","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8c90d8c3233ebbb75ee830d: start","job_status":"start","queue_duration_s":0.934144,"scheduling_latency_s":0.934158}1447gitlab # [ 165.773628] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.984Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8c90d8c3233ebbb75ee830d","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8c90d8c3233ebbb75ee830d: done: 0.008764 sec","job_status":"done","queue_duration_s":0.934144,"scheduling_latency_s":0.934158,"redis_calls":13,"redis_duration_s":0.0015170000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008764,"completed_at":"2026-10-10T11:01:04.984Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 165.780861] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.985Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"707d1b56b224e6f4eb9f473c","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-707d1b56b224e6f4eb9f473c: start","job_status":"start","queue_duration_s":0.943808,"scheduling_latency_s":0.943822}1449gitlab # [ 165.825587] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"707d1b56b224e6f4eb9f473c","created_at":"2026-10-10T11:01:03.936Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-6f522ee3c3664d5f","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fca28666f247ccc3ad20fd30bcb3c91e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.041Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-707d1b56b224e6f4eb9f473c: done: 0.008745 sec","job_status":"done","queue_duration_s":0.943808,"scheduling_latency_s":0.943822,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008745,"completed_at":"2026-10-10T11:01:04.994Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450gitlab # [ 165.826598] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:04.994Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0d8f1460f7560132796e751d","created_at":"2026-10-10T11:01:04.384Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-327760fc12474d8d","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d2df3f717c433da4e74110fbd6976e25","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.391Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0d8f1460f7560132796e751d: start","job_status":"start","queue_duration_s":0.603117,"scheduling_latency_s":0.603129}1451gitlab # [ 165.871316] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:05.005Z","message":"No existing merge request to be cleaned up."}1452gitlab # [ 165.871944] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:05.012Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0d8f1460f7560132796e751d","created_at":"2026-10-10T11:01:04.384Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-327760fc12474d8d","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d2df3f717c433da4e74110fbd6976e25","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.391Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0d8f1460f7560132796e751d: done: 0.017542 sec","job_status":"done","queue_duration_s":0.603117,"scheduling_latency_s":0.603129,"redis_calls":13,"redis_duration_s":0.001677,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000286,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000616,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000775,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000235226,"duration_s":0.017542,"completed_at":"2026-10-10T11:01:05.012Z","load_balancing_strategy":"primary","db_duration_s":0.002303,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 165.872823] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:05.013Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"83c766df19e25dc1b8853be6","created_at":"2026-10-10T11:01:04.384Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-327760fc12474d8d","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d2df3f717c433da4e74110fbd6976e25","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.391Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-83c766df19e25dc1b8853be6: start","job_status":"start","queue_duration_s":0.621558,"scheduling_latency_s":0.621571}1454gitlab # [ 165.913704] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:05.020Z","message":"No existing merge request to be cleaned up."}1455gitlab # [ 165.914555] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:05.024Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"83c766df19e25dc1b8853be6","created_at":"2026-10-10T11:01:04.384Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-327760fc12474d8d","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d2df3f717c433da4e74110fbd6976e25","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.391Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-83c766df19e25dc1b8853be6: done: 0.011114 sec","job_status":"done","queue_duration_s":0.621558,"scheduling_latency_s":0.621571,"redis_calls":10,"redis_duration_s":0.001219,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000729,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011114,"completed_at":"2026-10-10T11:01:05.024Z","load_balancing_strategy":"primary","db_duration_s":0.001239,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456gitlab # [ 165.918938] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:05.025Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f2d131ae70c80b02387000fc","created_at":"2026-10-10T11:01:04.384Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-327760fc12474d8d","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d2df3f717c433da4e74110fbd6976e25","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.391Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f2d131ae70c80b02387000fc: start","job_status":"start","queue_duration_s":0.633627,"scheduling_latency_s":0.63364}1457gitlab # [ 165.930094] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:05.031Z","message":"No existing merge request to be cleaned up."}1458gitlab # [ 165.931660] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:05.036Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f2d131ae70c80b02387000fc","created_at":"2026-10-10T11:01:04.384Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-327760fc12474d8d","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d2df3f717c433da4e74110fbd6976e25","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.391Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f2d131ae70c80b02387000fc: done: 0.0106 sec","job_status":"done","queue_duration_s":0.633627,"scheduling_latency_s":0.63364,"redis_calls":10,"redis_duration_s":0.001192,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000486,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000706,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0106,"completed_at":"2026-10-10T11:01:05.036Z","load_balancing_strategy":"primary","db_duration_s":0.000935,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 165.973062] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:05.037Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9e48b3d76fe7ecf2710f5838","created_at":"2026-10-10T11:01:04.384Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-327760fc12474d8d","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d2df3f717c433da4e74110fbd6976e25","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.391Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9e48b3d76fe7ecf2710f5838: start","job_status":"start","queue_duration_s":0.645187,"scheduling_latency_s":0.645202}1460gitlab # [ 165.977220] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:05.043Z","message":"No existing merge request to be cleaned up."}1461gitlab # [ 165.978692] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:05.047Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9e48b3d76fe7ecf2710f5838","created_at":"2026-10-10T11:01:04.384Z","trace_propagation_headers":{"sentry-trace":"1ca7fdc0c1764bf7b9c4c42f1a57a8cd-327760fc12474d8d","baggage":"sentry-trace_id=1ca7fdc0c1764bf7b9c4c42f1a57a8cd,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d2df3f717c433da4e74110fbd6976e25","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:04.391Z","job_size_bytes":2,"pid":1612,"sidekiq_tid":"maec","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9e48b3d76fe7ecf2710f5838: done: 0.010536 sec","job_status":"done","queue_duration_s":0.645187,"scheduling_latency_s":0.645202,"redis_calls":10,"redis_duration_s":0.001368,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000483,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000885,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010536,"completed_at":"2026-10-10T11:01:05.047Z","load_balancing_strategy":"primary","db_duration_s":0.000908,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462gitlab # [ 166.197545] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 1765558404 wd_nsec: 17655583121463gitlab # [ 182.358767] postgres[2429]: [2429] ERROR: relation "table_without_models" does not exist1464gitlab # [ 182.360356] postgres[2429]: [2429] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1465gitlab # [ 185.254184] runuser[2222]: pam_unix(runuser:session): session closed for user gitlab1466gitlab # [ 185.256288] setup-gitlab-start[2203]: + systemctl restart gitlab-sidekiq.service1467gitlab # [ 185.283508] systemd[1]: Stopping gitlab-sidekiq.service...1468gitlab # [ 185.285268] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:26.526Z","message":"Shutting down"}1469gitlab # [ 185.286138] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:26.527Z","message":"Scheduler exiting..."}1470gitlab # [ 185.286830] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:26.528Z","message":"Terminating quiet threads for default capsule"}1471gitlab # [ 185.287712] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:26.528Z","message":"Scheduler exiting..."}1472gitlab # [ 185.786541] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:27.029Z","message":"Pausing to allow jobs to finish..."}1473gitlab # [ 189.291448] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:30.534Z","message":"Bye!"}1474gitlab # [ 189.320140] sidekiq-cluster[1612]: {"severity":"INFO","time":"2026-10-10T11:01:30.563Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1612,"worker_id":"sidekiq_0","memwd_rss_bytes":939290624,"retry":0}1475gitlab # [ 190.286285] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1476gitlab # [ 190.288990] systemd[1]: Stopped gitlab-sidekiq.service.1477gitlab # [ 190.290857] systemd[1]: gitlab-sidekiq.service: Consumed 55.984s CPU time over 1min 57.061s wall clock time, 957.9M memory peak, 4.4M read from disk.1478gitlab # [ 190.300335] systemd[1]: Started gitlab-sidekiq.service.1479gitlab # [ 190.304096] setup-gitlab-start[2203]: + mkdir -p /var/lib/secrets1480gitlab # [ 190.307338] setup-gitlab-start[2203]: + echo nixbot-test-token-of-20-chars1481gitlab # [ 190.309545] setup-gitlab-start[2203]: + chmod 644 /var/lib/secrets/gitlab-token1482gitlab # [ 190.315310] setup-gitlab-start[2203]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1483gitlab # [ 190.583175] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:31 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 2271484gitlab # [ 191.061629] gitaly[904]: time="2026-10-10T11:01:32.304Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T11:01:42.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.304" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1485gitlab # [ 191.062899] gitaly[904]: time="2026-10-10T11:01:32.304Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T11:01:42.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.304" grpc.time_ms=0.445 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1486gitlab # [ 191.143738] gitaly[904]: time="2026-10-10T11:01:32.387Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T11:01:42.086" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.386" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1487gitlab # [ 191.145504] gitaly[904]: time="2026-10-10T11:01:32.387Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T11:01:42.086" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.386" grpc.time_ms=0.367 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1488gitlab # [ 191.238132] gitaly[904]: time="2026-10-10T11:01:32.481Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW diskcache=5c1374f3-6f3a-4b5c-a6c9-01e88afdc4cf grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-10T11:01:42.137" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.437" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1489gitlab # [ 191.257117] gitaly[904]: time="2026-10-10T11:01:32.500Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1573 command.oublock=32 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-10T11:01:42.137" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.437" grpc.time_ms=44.048 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1490gitlab # [ 191.282667] gitaly[904]: time="2026-10-10T11:01:32.526Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T11:01:42.224" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.525" pid=904 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1491gitlab # [ 191.283805] gitaly[904]: time="2026-10-10T11:01:32.526Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T11:01:42.224" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.525" grpc.time_ms=0.86 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1492gitlab # [ 191.316406] gitaly[904]: time="2026-10-10T11:01:32.559Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T11:01:42.248" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.548" grpc.time_ms=10.86 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1493gitlab # [ 191.401827] gitaly[904]: time="2026-10-10T11:01:32.645Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T11:01:42.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.645" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1494gitlab # [ 191.403670] gitaly[904]: time="2026-10-10T11:01:32.645Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T11:01:42.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.645" grpc.time_ms=0.26 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1495gitlab # [ 191.492070] gitaly[904]: time="2026-10-10T11:01:32.735Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW diskcache=b42d3d91-2cd8-4ab5-adf1-39dc20817cf8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-10T11:01:42.048" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.688" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1496gitlab # [ 191.511062] gitaly[904]: time="2026-10-10T11:01:32.735Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1572 command.oublock=32 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-10T11:01:42.048" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.688" grpc.time_ms=47.091 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1497gitlab # [ 191.512504] gitaly[904]: time="2026-10-10T11:01:32.743Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T11:01:42.102" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.742" pid=904 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1498gitlab # [ 191.513902] gitaly[904]: time="2026-10-10T11:01:32.743Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T11:01:42.102" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.742" grpc.time_ms=0.485 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1499gitlab # [ 191.515310] gitaly[904]: time="2026-10-10T11:01:32.751Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T11:01:42.105" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.746" grpc.time_ms=5.527 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1500gitlab # [ 191.588753] gitaly[904]: time="2026-10-10T11:01:32.832Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T11:01:42.191" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.831" grpc.time_ms=0.235 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1501gitlab # [ 191.600383] gitaly[904]: time="2026-10-10T11:01:32.843Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-10T11:01:42.202" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:32.843" grpc.time_ms=0.422 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1502gitlab # [ 192.118418] gitaly[904]: time="2026-10-10T11:01:33.361Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQK9HZZ0DWXY545BJATFAW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T11:01:43.054" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:33.355" grpc.time_ms=5.936 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1503gitlab # [ 192.344739] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:33 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 17481504gitlab # [ 192.352635] setup-gitlab-start[2488]: {"id":1,"message":null,"error":null}1505gitlab # [ 193.339993] sidekiq-cluster[2476]: {"severity":"INFO","time":"2026-10-10T11:01:34.566Z","message":"Starting cluster with 1 processes"}1506gitlab # [ 197.488310] gitaly[904]: time="2026-10-10T11:01:38.731Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKG61THN4WC7N6T76AC7W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T11:01:48.079" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:38.720" grpc.time_ms=11.243 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1507gitlab # [ 197.554691] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:38 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 1711508gitlab # [ 197.559233] setup-gitlab-start[2203]: + mkdir -p /tmp/test-flake1509gitlab # [ 197.562422] setup-gitlab-start[2203]: + cd /tmp/test-flake1510gitlab # [ 197.563249] setup-gitlab-start[2203]: + git init -b master1511gitlab # [ 197.591379] setup-gitlab-start[2561]: Initialized empty Git repository in /tmp/test-flake/.git/1512gitlab # [ 197.592618] setup-gitlab-start[2203]: + git config user.name test1513gitlab # [ 197.598057] setup-gitlab-start[2203]: + git config user.email test@example.com1514gitlab # [ 197.603412] setup-gitlab-start[2203]: + cat1515gitlab # [ 197.607523] setup-gitlab-start[2203]: + git add flake.nix1516gitlab # [ 197.612707] setup-gitlab-start[2203]: + git commit -m 'initial commit'1517gitlab # [ 197.640571] setup-gitlab-start[2566]: [master (root-commit) fa8a9f4] initial commit1518gitlab # [ 197.643784] setup-gitlab-start[2566]: 1 file changed, 10 insertions(+)1519gitlab # [ 197.645966] setup-gitlab-start[2566]: create mode 100644 flake.nix1520gitlab # [ 197.648203] setup-gitlab-start[2203]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1521gitlab # [ 197.652492] setup-gitlab-start[2203]: + git push -u origin master1522gitlab # [ 197.998635] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:39 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 3131523gitlab # [ 198.101086] gitaly[904]: time="2026-10-10T11:01:39.344Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGSFCF4SZ21SJ0WSECKK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:39.333" grpc.time_ms=10.523 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1524gitlab # [ 198.131177] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:39 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1271525gitlab # [ 198.581183] gitaly[904]: time="2026-10-10T11:01:39.824Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T11:01:49.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:39.812" grpc.time_ms=11.467 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1526gitlab # [ 198.622788] gitaly[904]: time="2026-10-10T11:01:39.866Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-10T11:02:09.053" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-10T11:01:39.854" grpc.time_ms=11.98 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1527gitlab # [ 198.645192] gitaly[904]: time="2026-10-10T11:01:39.888Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B duration_ms=152ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/allowed"1528gitlab # [ 198.670394] gitaly[904]: time="2026-10-10T11:01:39.913Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B duration_ms=25ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/pre_receive"1529gitlab # [ 198.675835] gitaly[904]: time="2026-10-10T11:01:39.919Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5306 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:01:39.734" grpc.time_ms=184.47 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1530gitlab # [ 198.711775] gitaly[904]: time="2026-10-10T11:01:39.955Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4776 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:01:39.954" grpc.time_ms=0.456 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1531gitlab # [ 198.748224] gitaly[904]: time="2026-10-10T11:01:39.991Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4775 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:01:39.990" grpc.time_ms=1.071 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1532gitlab # [ 198.789646] gitaly[904]: time="2026-10-10T11:01:40.033Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4777 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:01:40.031" grpc.time_ms=0.913 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1533gitlab # [ 198.871092] gitaly[904]: time="2026-10-10T11:01:40.114Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T11:01:50.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:40.103" grpc.time_ms=10.962 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1534gitlab # [ 198.914679] gitaly[904]: time="2026-10-10T11:01:40.158Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-10T11:01:50.037" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:01:40.138" grpc.time_ms=19.836 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1535gitlab # [ 198.928340] gitaly[904]: time="2026-10-10T11:01:40.171Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B duration_ms=108ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/post_receive"1536gitlab # [ 198.929370] gitaly[904]: time="2026-10-10T11:01:40.172Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4800 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:01:40.063" grpc.time_ms=109.316 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1537gitlab # [ 198.952644] gitaly[904]: time="2026-10-10T11:01:40.196Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B diskcache=30df37d5-e740-4366-8656-edac9413f9c0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:39.650" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1538gitlab # [ 198.953779] gitaly[904]: time="2026-10-10T11:01:40.196Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=194 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13762 command.oublock=32 command.real_time_ms=533 command.spawn_token_fork_ms=1 command.system_time_ms=130 command.user_time_ms=64 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:39.650" grpc.time_ms=545.33 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1539gitlab # [ 198.954974] gitlab-workhorse[825]: time="2026-10-10T11:01:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521540gitlab # [ 198.955754] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:40 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 8061541gitlab # [ 198.978937] setup-gitlab-start[2570]: To http://gitlab/root/test-flake.git1542gitlab # [ 198.980092] setup-gitlab-start[2570]: * [new branch] master -> master1543gitlab # [ 198.984190] setup-gitlab-start[2570]: branch 'master' set up to track 'origin/master'.1544gitlab # [ 198.988273] systemd[1]: Finished setup-gitlab.service.1545gitlab # [ 198.990575] systemd[1]: Starting nixbot CI service...1546gitlab: (finished: waiting for unit setup-gitlab.service, in 200.15 seconds)1547gitlab: waiting for unit nixbot.service1548gitlab # [ 200.310273] nixbot[2659]: {"ts": "2026-10-10T11:01:41.552Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1549gitlab # [ 200.343692] nixbot[2659]: {"ts": "2026-10-10T11:01:41.587Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1550gitlab # [ 200.376565] nixbot[2659]: {"ts": "2026-10-10T11:01:41.619Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1551gitlab # [ 200.387118] nixbot[2659]: {"ts": "2026-10-10T11:01:41.630Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1552gitlab # [ 200.398132] nixbot[2659]: {"ts": "2026-10-10T11:01:41.641Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1553gitlab # [ 200.412137] nixbot[2659]: {"ts": "2026-10-10T11:01:41.655Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1554gitlab # [ 200.422273] nixbot[2659]: {"ts": "2026-10-10T11:01:41.665Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1555gitlab # [ 200.429797] nixbot[2659]: {"ts": "2026-10-10T11:01:41.673Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1556gitlab # [ 200.443245] nixbot[2659]: {"ts": "2026-10-10T11:01:41.686Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1557gitlab # [ 200.451431] nixbot[2659]: {"ts": "2026-10-10T11:01:41.694Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1558gitlab # [ 200.462979] nixbot[2659]: {"ts": "2026-10-10T11:01:41.706Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1559gitlab # [ 200.472419] nixbot[2659]: {"ts": "2026-10-10T11:01:41.715Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1560gitlab # [ 200.481773] nixbot[2659]: {"ts": "2026-10-10T11:01:41.725Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1561gitlab # [ 200.490340] nixbot[2659]: {"ts": "2026-10-10T11:01:41.733Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1562gitlab # [ 200.500977] nixbot[2659]: {"ts": "2026-10-10T11:01:41.744Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1563gitlab # [ 200.509267] nixbot[2659]: {"ts": "2026-10-10T11:01:41.752Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1564gitlab # [ 200.516720] nixbot[2659]: {"ts": "2026-10-10T11:01:41.760Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1565gitlab # [ 200.523955] nixbot[2659]: {"ts": "2026-10-10T11:01:41.767Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1566gitlab # [ 200.535335] nixbot[2659]: {"ts": "2026-10-10T11:01:41.778Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1567gitlab # [ 200.543164] nixbot[2659]: {"ts": "2026-10-10T11:01:41.786Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1568gitlab # [ 200.549570] nixbot[2659]: {"ts": "2026-10-10T11:01:41.793Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1569gitlab # [ 200.560220] nixbot[2659]: {"ts": "2026-10-10T11:01:41.803Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1570gitlab # [ 200.566558] nixbot[2659]: {"ts": "2026-10-10T11:01:41.809Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1571gitlab # [ 200.573565] nixbot[2659]: {"ts": "2026-10-10T11:01:41.816Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1572gitlab # [ 200.577710] nixbot[2659]: {"ts": "2026-10-10T11:01:41.821Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1573gitlab # [ 200.582409] nixbot[2659]: {"ts": "2026-10-10T11:01:41.825Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1574gitlab # [ 200.586671] nixbot[2659]: {"ts": "2026-10-10T11:01:41.830Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1575gitlab # [ 200.590747] nixbot[2659]: {"ts": "2026-10-10T11:01:41.834Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1576gitlab # [ 200.594853] nixbot[2659]: {"ts": "2026-10-10T11:01:41.838Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1577gitlab # [ 200.605438] nixbot[2659]: {"ts": "2026-10-10T11:01:41.848Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1578gitlab # [ 200.611331] nixbot[2659]: {"ts": "2026-10-10T11:01:41.854Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1579gitlab # [ 200.615457] nixbot[2659]: {"ts": "2026-10-10T11:01:41.858Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1580gitlab # [ 200.619462] nixbot[2659]: {"ts": "2026-10-10T11:01:41.862Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1581gitlab # [ 200.628446] nixbot[2659]: {"ts": "2026-10-10T11:01:41.871Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1582gitlab # [ 200.633114] nixbot[2659]: {"ts": "2026-10-10T11:01:41.876Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1583gitlab # [ 200.644418] nixbot[2659]: {"ts": "2026-10-10T11:01:41.887Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1584gitlab # [ 200.651236] nixbot[2659]: {"ts": "2026-10-10T11:01:41.894Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1585gitlab # [ 200.656806] nixbot[2659]: {"ts": "2026-10-10T11:01:41.900Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "effect_when_changed"}1586gitlab # [ 201.060087] nixbot[2659]: {"ts": "2026-10-10T11:01:42.303Z", "level": "info", "logger": "uvicorn.error", "msg": "Started server process [2659]", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"}1587gitlab # [ 201.065755] nixbot[2659]: {"ts": "2026-10-10T11:01:42.303Z", "level": "info", "logger": "uvicorn.error", "msg": "Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)"}1588gitlab # [ 201.081474] nixbot[2659]: {"ts": "2026-10-10T11:01:42.324Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:54112 - \"GET /health HTTP/1.1\" 200"}1589gitlab # [ 201.089925] systemd[1]: Started nixbot CI service.1590gitlab # [ 201.091033] systemd[1]: Reached target Multi-User System.1591gitlab # [ 201.092940] systemd[1]: Startup finished in 1.199s (kernel) + 3.610s (initrd) + 3min 16.282s (userspace) = 3min 21.092s.1592gitlab: (finished: waiting for unit nixbot.service, in 2.09 seconds)1593gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1594gitlab # [ 201.636600] nixbot[2659]: {"ts": "2026-10-10T11:01:42.878Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:54124 - \"GET /health HTTP/1.1\" 200"}1595gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1596subtest: project discovered and webhook registered1597gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1598gitlab # [ 201.735989] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:42 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 601599gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds)1600gitlab # [ 201.770493] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:43 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 6921601gitlab # [ 201.785484] nixbot[2659]: {"ts": "2026-10-10T11:01:43.028Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1602gitlab # [ 201.846247] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 531603gitlab # [ 201.854412] nixbot[2659]: {"ts": "2026-10-10T11:01:43.097Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1604gitlab # [ 201.994797] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 1331605gitlab # [ 202.107530] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:43 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1061606gitlab # [ 202.240889] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 1151607gitlab # [ 202.298070] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:43 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 441608gitlab # [ 202.362118] gitaly[904]: time="2026-10-10T11:01:43.604Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKN01V317K0TGXP568CBE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-10-10T11:02:13.088" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:01:43.588" grpc.time_ms=15.821 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1609gitlab # [ 202.406486] gitaly[904]: time="2026-10-10T11:01:43.649Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKN01V317K0TGXP568CBE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:02:13.147" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:01:43.648" grpc.time_ms=1.021 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1610gitlab # [ 202.442585] gitaly[904]: time="2026-10-10T11:01:43.686Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKN01V317K0TGXP568CBE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-10-10T11:01:53.036" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:01:43.676" grpc.time_ms=9.325 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1611gitlab # [ 202.465952] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1551612gitlab # [ 202.525675] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 541613gitlab # [ 202.531644] nixbot[2659]: {"ts": "2026-10-10T11:01:43.775Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1614gitlab # [ 202.660904] gitaly[904]: time="2026-10-10T11:01:43.904Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKN9EHGDHF93TDTTDZ3HC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:43.894" grpc.time_ms=9.679 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1615gitlab # [ 202.690870] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:43 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 791616gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1617gitlab # [ 202.794276] gitaly[904]: time="2026-10-10T11:01:44.037Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKNC3QBTDA27D5X6B5SJS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:44.029" pid=904 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1618gitlab # [ 202.795477] gitaly[904]: time="2026-10-10T11:01:44.037Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKNC3QBTDA27D5X6B5SJS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:44.029" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1619gitlab # [ 202.796544] gitlab-workhorse[825]: time="2026-10-10T11:01:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131620gitlab # [ 202.797632] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 991621gitlab # [ 202.799077] gitaly[904]: time="2026-10-10T11:01:44.041Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKNC3QBTDA27D5X6B5SJS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:44.029" grpc.time_ms=9.279 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1622gitlab # [ 202.866221] gitaly[904]: time="2026-10-10T11:01:44.109Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKNFG595QAMBWMNWMH1J0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:44.086" pid=904 remote_ip="2001:db8:1::1" request_sha=6ac054c667074ad60a4d9d886bfeed58d4a0306a response_bytes=228 span.kind=server system=grpc1623gitlab # [ 202.866834] gitaly[904]: time="2026-10-10T11:01:44.109Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1203 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKNFG595QAMBWMNWMH1J0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:44.086" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1624gitlab # [ 202.867526] gitaly[904]: time="2026-10-10T11:01:44.110Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1203 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKNFG595QAMBWMNWMH1J0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:44.086" grpc.time_ms=23.398 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1625gitlab # [ 202.868216] gitlab-workhorse[825]: time="2026-10-10T11:01:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2281626gitlab # [ 202.868632] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 228 "" "git/2.55.0" 621627gitlab # [ 202.886826] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 1151628gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.17 seconds)1629(finished: subtest: project discovered and webhook registered, in 1.27 seconds)1630subtest: push delivers the webhook1631gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1632gitlab # [ 202.975613] gitaly[904]: time="2026-10-10T11:01:44.219Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKNKJN2N3QA4KVE8KYNAT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=194 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:44.210" grpc.time_ms=8.982 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1633gitlab # [ 202.977222] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:44 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 411634gitlab # [ 203.043494] gitaly[904]: time="2026-10-10T11:01:44.286Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKNN9KT0B0GA3F9YA007V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:44.276" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1635gitlab # [ 203.061261] gitaly[904]: time="2026-10-10T11:01:44.287Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKNN9KT0B0GA3F9YA007V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:44.276" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1636gitlab # [ 203.062832] gitlab-workhorse[825]: time="2026-10-10T11:01:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671637gitlab # [ 203.063714] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 531638gitlab # [ 203.064612] gitaly[904]: time="2026-10-10T11:01:44.287Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKNN9KT0B0GA3F9YA007V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:44.276" grpc.time_ms=10.646 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1639gitlab # [ 203.162542] gitaly[904]: time="2026-10-10T11:01:44.406Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKNSCY3GARFA9MTCE60AJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:44.396" grpc.time_ms=9.117 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1640gitlab # [ 203.185284] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:44 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 631641gitlab # [ 203.259781] gitaly[904]: time="2026-10-10T11:01:44.503Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKNVFKE3CF953STR7X5RJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:44.488" pid=904 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1642gitlab # [ 203.261339] gitaly[904]: time="2026-10-10T11:01:44.503Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1171 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKNVFKE3CF953STR7X5RJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:44.488" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1643gitlab # [ 203.306246] gitaly[904]: time="2026-10-10T11:01:44.549Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1171 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKNVFKE3CF953STR7X5RJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:01:44.488" grpc.time_ms=14.914 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1644gitlab # [ 203.307946] gitlab-workhorse[825]: time="2026-10-10T11:01:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091645gitlab # [ 203.308553] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 1191646gitlab # [ 203.394067] gitaly[904]: time="2026-10-10T11:01:44.636Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:02:14.035" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:01:44.635" grpc.time_ms=0.857 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1647gitlab # [ 203.454688] gitaly[904]: time="2026-10-10T11:01:44.698Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-10T11:02:14.090" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:01:44.691" grpc.time_ms=6.807 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1648gitlab # [ 203.494143] gitaly[904]: time="2026-10-10T11:01:44.737Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:01:54.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:01:44.731" grpc.time_ms=5.907 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1649gitlab # [ 203.517233] gitaly[904]: time="2026-10-10T11:01:44.760Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:01:54.024" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:01:44.755" grpc.time_ms=5.51 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1650gitlab # [ 203.594161] gitaly[904]: time="2026-10-10T11:01:44.837Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-10T11:01:54.026" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:44.837" grpc.time_ms=0.213 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1651gitlab # [ 203.630661] gitaly[904]: time="2026-10-10T11:01:44.874Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-10T11:01:54.056" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:01:44.867" grpc.time_ms=6.794 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1652gitlab # [ 203.699525] gitaly[904]: time="2026-10-10T11:01:44.943Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4146 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:01:44.941" grpc.time_ms=1.288 pid=904 span.kind=server system=grpc1653gitlab # [ 203.737291] gitaly[904]: time="2026-10-10T11:01:44.980Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:01:44.980" grpc.time_ms=0.336 pid=904 span.kind=server system=grpc1654gitlab # [ 203.756601] gitaly[904]: time="2026-10-10T11:01:45.000Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS diskcache=d82f8aa5-fe21-4253-b2bf-d0bbc7cf3d7c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T11:01:54.090" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:44.900" pid=904 ref=refs/keep-around/fa8a9f48328d3d85c52fff7086159d5273f9ff52 remote_ip="2001:db8:1::1" revision=fa8a9f48328d3d85c52fff7086159d5273f9ff52 span.kind=server system=grpc username=root1655gitlab # [ 203.757553] gitaly[904]: time="2026-10-10T11:01:45.000Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=66 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5498 command.oublock=8 command.real_time_ms=90 command.spawn_token_fork_ms=0 command.system_time_ms=44 command.user_time_ms=22 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T11:01:54.090" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:01:44.900" grpc.time_ms=99.769 pid=904 ref=refs/keep-around/fa8a9f48328d3d85c52fff7086159d5273f9ff52 remote_ip="2001:db8:1::1" revision=fa8a9f48328d3d85c52fff7086159d5273f9ff52 span.kind=server system=grpc username=root1656gitlab # [ 203.761324] gitaly[904]: time="2026-10-10T11:01:45.004Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:02:15.003" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:01:45.004" grpc.time_ms=0.617 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1657gitlab # [ 203.909699] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:45 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fa8a9f48328d3d85c52fff7086159d5273f9ff52 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 5601658gitlab # [ 203.927306] nixbot[2659]: {"ts": "2026-10-10T11:01:45.170Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-fa8a9f48328d-0"}1659gitlab # [ 204.092054] nixbot[2659]: {"ts": "2026-10-10T11:01:45.335Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-fa8a9f48328d-0", "argv0": "bwrap"}1660gitlab # [ 204.194534] systemd[1]: Started Nix Daemon.1661gitlab # [ 204.267321] nix-daemon[2824]: accepted connection from pid 2819, user nixbot1662gitlab # [ 209.275826] nix-daemon[2824]: accepted connection from pid 2873, user nixbot1663gitlab # [ 209.287202] nix-daemon[2824]: accepted connection from pid 2819, user nixbot1664gitlab # [ 214.183408] nix-daemon[2824]: accepted connection from pid 2819, user nixbot1665gitlab # [ 214.282762] nix-daemon[2824]: accepted connection from pid 2916, user nixbot1666gitlab # [ 214.339370] gitaly[904]: time="2026-10-10T11:01:55.581Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQM0NSA3TWT9H0STRDGZ6Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:02:25.577" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:01:55.577" grpc.time_ms=3.212 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1667gitlab # [ 214.362352] gitaly[904]: time="2026-10-10T11:01:55.605Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQM0NSA3TWT9H0STRDGZ6Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:05.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:01:55.600" grpc.time_ms=5.023 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1668gitlab # [ 214.396026] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fa8a9f48328d3d85c52fff7086159d5273f9ff52 HTTP/1.1" 201 676 "" "python-httpx/0.28.1" 1251669gitlab # [ 214.436427] gitaly[904]: time="2026-10-10T11:01:55.679Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQM0SWSGJWEE3WWEX0FFPT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:02:25.078" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:01:55.678" grpc.time_ms=0.726 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1670gitlab # [ 214.486874] gitaly[904]: time="2026-10-10T11:01:55.730Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQM0SWSGJWEE3WWEX0FFPT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:05.133" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:01:55.723" grpc.time_ms=6.789 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1671gitlab # [ 214.568943] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:01:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fa8a9f48328d3d85c52fff7086159d5273f9ff52 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1671672gitlab # [ 218.032687] sidekiq-cluster[2533]: 2026-10-10T11:01:59.275Z pid=2533 tid=xu5 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2533, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>448073728}1673gitlab # [ 218.199350] sidekiq-cluster[2533]: W, [2026-10-10T11:01:59.442679 #2533] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1674gitlab # [ 218.236418] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:01:59.479Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1675gitlab # [ 219.017222] nix-daemon[2824]: accepted connection from pid 2993, user nixbot1676gitlab # [ 219.092704] gitaly[904]: time="2026-10-10T11:02:00.335Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQM5B9D4181F7F3XJ7SF3K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:02:30.333" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:00.334" grpc.time_ms=0.772 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1677gitlab # [ 219.148775] gitaly[904]: time="2026-10-10T11:02:00.392Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQM5B9D4181F7F3XJ7SF3K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:10.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:00.381" grpc.time_ms=10.818 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1678gitlab # [ 219.184934] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:00 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fa8a9f48328d3d85c52fff7086159d5273f9ff52 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1291679gitlab # [ 219.423224] nixbot[2659]: {"ts": "2026-10-10T11:02:00.665Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 1, "sha": "fa8a9f48328d3d85c52fff7086159d5273f9ff52", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []}1680gitlab # [ 219.549687] gitaly[904]: time="2026-10-10T11:02:00.793Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQM5RKMGF8JHG2H5MJQ0V4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:00.784" grpc.time_ms=8.309 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1681gitlab # [ 219.551415] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:00 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 701682gitlab # [ 219.655135] gitaly[904]: time="2026-10-10T11:02:00.898Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQM5TXA8G7P4NY59Q1ADXC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:00.885" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1683gitlab # [ 219.655878] gitaly[904]: time="2026-10-10T11:02:00.898Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQM5TXA8G7P4NY59Q1ADXC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:00.885" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1684gitlab # [ 219.657638] gitaly[904]: time="2026-10-10T11:02:00.900Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQM5TXA8G7P4NY59Q1ADXC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:00.885" grpc.time_ms=13.874 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1685gitlab # [ 219.660045] gitlab-workhorse[825]: time="2026-10-10T11:02:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671686gitlab # [ 219.661118] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1031687gitlab # [ 219.781392] gitaly[904]: time="2026-10-10T11:02:01.024Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQM60MDMTCGNN5HAQ0FBPY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:01.017" grpc.time_ms=6.962 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1688gitlab # [ 219.783699] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 461689gitlab # [ 219.820989] gitaly[904]: time="2026-10-10T11:02:01.064Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQM62432WRJQKB4RAAF5RE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:01.058" pid=904 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1690gitlab # [ 219.821773] gitaly[904]: time="2026-10-10T11:02:01.064Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQM62432WRJQKB4RAAF5RE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:01.058" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1691gitlab # [ 219.842069] gitaly[904]: time="2026-10-10T11:02:01.085Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQM62432WRJQKB4RAAF5RE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:01.058" grpc.time_ms=26.842 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1692gitlab # [ 219.855505] gitlab-workhorse[825]: time="2026-10-10T11:02:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671693gitlab # [ 219.856203] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 691694gitlab # [ 220.334330] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:01.577Z","message":"GitLab reliable fetch activated!"}1695gitlab # [ 220.336299] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:01.577Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1696gitlab # [ 239.809335] postgres[3205]: [3205] ERROR: relation "table_without_models" does not exist1697gitlab # [ 239.810789] postgres[3205]: [3205] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1698gitlab # [ 242.165499] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:23.408Z","message":"Booted Rails 7.2.3.1 application in production environment"}1699gitlab # [ 242.169128] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:23.412Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1700gitlab # [ 242.169961] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:23.412Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1701gitlab # [ 242.171113] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:23.412Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1702gitlab # [ 242.172964] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:23.416Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1703gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 39.46 seconds)1704gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1705gitlab # [ 242.472054] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:23 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 391706gitlab # [ 242.557323] gitaly[904]: time="2026-10-10T11:02:23.800Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JQMW76C76DQQKEXHXVPNNZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:23.786" grpc.time_ms=13.192 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1707gitlab # [ 242.558747] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:23 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 811708gitlab # Enumerating objects: 5, done.1709gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1710gitlab # Delta compression using up to 4 threads1711gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1712gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 137.00 KiB/s, done.1713gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1714gitlab # [ 242.840230] gitaly[904]: time="2026-10-10T11:02:24.083Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-10T11:02:54.070" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-10T11:02:24.071" grpc.time_ms=11.919 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1715gitlab # [ 242.900836] gitaly[904]: time="2026-10-10T11:02:24.144Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C duration_ms=133ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/allowed"1716gitlab # [ 242.938474] gitaly[904]: time="2026-10-10T11:02:24.181Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C duration_ms=30ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/pre_receive"1717gitlab # [ 242.945549] gitaly[904]: time="2026-10-10T11:02:24.189Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5306 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:24.010" grpc.time_ms=178.744 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1718gitlab # [ 243.007236] gitaly[904]: time="2026-10-10T11:02:24.250Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4776 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:24.249" grpc.time_ms=1.179 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1719gitlab # [ 243.054583] gitaly[904]: time="2026-10-10T11:02:24.298Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4775 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:24.297" grpc.time_ms=0.53 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1720gitlab # [ 243.102928] gitaly[904]: time="2026-10-10T11:02:24.346Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4777 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:24.345" grpc.time_ms=0.443 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1721gitlab # [ 243.203841] gitaly[904]: time="2026-10-10T11:02:24.447Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C duration_ms=63ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/post_receive"1722gitlab # [ 243.210854] gitaly[904]: time="2026-10-10T11:02:24.454Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4800 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:24.382" grpc.time_ms=71.702 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1723gitlab # [ 243.244284] gitaly[904]: time="2026-10-10T11:02:24.487Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C diskcache=a7f54827-cd8e-4470-a471-2902fe917033 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:23.932" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1724gitlab # [ 243.261342] gitaly[904]: time="2026-10-10T11:02:24.504Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=266 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13635 command.oublock=32 command.real_time_ms=545 command.spawn_token_fork_ms=0 command.system_time_ms=187 command.user_time_ms=79 component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:23.932" grpc.time_ms=555.44 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1725gitlab # [ 243.262068] gitlab-workhorse[825]: time="2026-10-10T11:02:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521726gitlab # [ 243.262858] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:24 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 6661727gitlab # To http://gitlab/root/test-flake.git1728gitlab # fa8a9f4..8827c58 master -> master1729gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.91 seconds)1730gitlab: must succeed: git -C /tmp/test-flake rev-parse master1731gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1732gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b'1733gitlab # [ 243.368895] nixbot[2659]: {"ts": "2026-10-10T11:02:24.611Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35532 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b HTTP/1.1\" 200"}1734gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b', in 0.08 seconds)1735gitlab # [ 243.521624] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:24.763Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a6d9cbdef6b5d694c33b837d","created_at":"2026-10-10T11:01:32.583Z","trace_propagation_headers":{"sentry-trace":"86c8ea7f1ae344faa892a74264cc672b-d5bda043f2c34112","baggage":"sentry-trace_id=86c8ea7f1ae344faa892a74264cc672b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQK9HZZ0DWXY545BJATFAW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:32.624Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a6d9cbdef6b5d694c33b837d: start","job_status":"start","queue_duration_s":52.139607,"scheduling_latency_s":52.13967}1736gitlab # [ 243.754844] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:24.998Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a6d9cbdef6b5d694c33b837d","created_at":"2026-10-10T11:01:32.583Z","trace_propagation_headers":{"sentry-trace":"86c8ea7f1ae344faa892a74264cc672b-d5bda043f2c34112","baggage":"sentry-trace_id=86c8ea7f1ae344faa892a74264cc672b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQK9HZZ0DWXY545BJATFAW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:32.624Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a6d9cbdef6b5d694c33b837d: done: 0.23445 sec","job_status":"done","queue_duration_s":52.139607,"scheduling_latency_s":52.13967,"redis_calls":24,"redis_duration_s":0.007807000000000001,"redis_read_bytes":1038,"redis_write_bytes":2604,"redis_feature_flag_calls":14,"redis_feature_flag_duration_s":0.004217,"redis_feature_flag_read_bytes":1030,"redis_feature_flag_write_bytes":1439,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002036,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001554,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.086041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.096656698,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.23445,"completed_at":"2026-10-10T11:02:24.997Z","load_balancing_strategy":"primary","db_duration_s":0.062009,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1737gitlab # [ 243.829701] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:25.072Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"8bf30ec8a392ae174a8aad80","created_at":"2026-10-10T11:01:32.637Z","trace_propagation_headers":{"sentry-trace":"86c8ea7f1ae344faa892a74264cc672b-d5bda043f2c34112","baggage":"sentry-trace_id=86c8ea7f1ae344faa892a74264cc672b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQK9HZZ0DWXY545BJATFAW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:32.641Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-8bf30ec8a392ae174a8aad80: start","job_status":"start","queue_duration_s":52.431326,"scheduling_latency_s":52.431393}1738gitlab # [ 243.951943] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:25.195Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"8bf30ec8a392ae174a8aad80","created_at":"2026-10-10T11:01:32.637Z","trace_propagation_headers":{"sentry-trace":"86c8ea7f1ae344faa892a74264cc672b-d5bda043f2c34112","baggage":"sentry-trace_id=86c8ea7f1ae344faa892a74264cc672b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQK9HZZ0DWXY545BJATFAW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:32.641Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-8bf30ec8a392ae174a8aad80: done: 0.122681 sec","job_status":"done","queue_duration_s":52.431326,"scheduling_latency_s":52.431393,"redis_calls":14,"redis_duration_s":0.0032649999999999997,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000636,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001571,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.085069192,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.122681,"completed_at":"2026-10-10T11:02:25.195Z","load_balancing_strategy":"primary","db_duration_s":0.025637,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1739gitlab # [ 243.954613] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:25.196Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"996b94d619e0de5717b5aeec","created_at":"2026-10-10T11:01:33.226Z","trace_propagation_headers":{"sentry-trace":"86c8ea7f1ae344faa892a74264cc672b-d5bda043f2c34112","baggage":"sentry-trace_id=86c8ea7f1ae344faa892a74264cc672b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQK9HZZ0DWXY545BJATFAW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:33.232Z","job_size_bytes":5,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-996b94d619e0de5717b5aeec: start","job_status":"start","queue_duration_s":51.964422,"scheduling_latency_s":51.964438}1740gitlab # [ 244.184926] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:25.427Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"996b94d619e0de5717b5aeec","created_at":"2026-10-10T11:01:33.226Z","trace_propagation_headers":{"sentry-trace":"86c8ea7f1ae344faa892a74264cc672b-d5bda043f2c34112","baggage":"sentry-trace_id=86c8ea7f1ae344faa892a74264cc672b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQK9HZZ0DWXY545BJATFAW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:33.232Z","job_size_bytes":5,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-996b94d619e0de5717b5aeec: done: 0.231318 sec","job_status":"done","queue_duration_s":51.964422,"scheduling_latency_s":51.964438,"redis_calls":12,"redis_duration_s":0.0032400000000000003,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00095,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001873,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.092,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.092,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.119718,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029026,"duration_s":0.231318,"completed_at":"2026-10-10T11:02:25.427Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.103861,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1741gitlab # [ 244.263849] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:25.506Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"cf5b5a7029e1eb7b3d340814","created_at":"2026-10-10T11:01:33.253Z","trace_propagation_headers":{"sentry-trace":"86c8ea7f1ae344faa892a74264cc672b-d5bda043f2c34112","baggage":"sentry-trace_id=86c8ea7f1ae344faa892a74264cc672b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQK9HZZ0DWXY545BJATFAW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:33.257Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-cf5b5a7029e1eb7b3d340814: start","job_status":"start","queue_duration_s":52.24988,"scheduling_latency_s":52.249913}1742gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b'1743gitlab # [ 244.412231] nixbot[2659]: {"ts": "2026-10-10T11:02:25.654Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35548 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b HTTP/1.1\" 200"}1744gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b', in 0.04 seconds)1745gitlab # [ 244.519682] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:25.762Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"cf5b5a7029e1eb7b3d340814","created_at":"2026-10-10T11:01:33.253Z","trace_propagation_headers":{"sentry-trace":"86c8ea7f1ae344faa892a74264cc672b-d5bda043f2c34112","baggage":"sentry-trace_id=86c8ea7f1ae344faa892a74264cc672b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQK9HZZ0DWXY545BJATFAW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:33.257Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-cf5b5a7029e1eb7b3d340814: done: 0.256021 sec","job_status":"done","queue_duration_s":52.24988,"scheduling_latency_s":52.249913,"redis_calls":11,"redis_duration_s":0.00269,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000713,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001479,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000498,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.198,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.198,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.157028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000336076,"duration_s":0.256021,"completed_at":"2026-10-10T11:02:25.762Z","load_balancing_strategy":"primary","db_duration_s":0.086891,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1746gitlab # [ 244.594229] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:25.764Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"d469e3dae4d7e9b9abefd5ef","created_at":"2026-10-10T11:01:33.259Z","trace_propagation_headers":{"sentry-trace":"86c8ea7f1ae344faa892a74264cc672b-d5bda043f2c34112","baggage":"sentry-trace_id=86c8ea7f1ae344faa892a74264cc672b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQK9HZZ0DWXY545BJATFAW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:33.262Z","job_size_bytes":89,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-d469e3dae4d7e9b9abefd5ef: start","job_status":"start","queue_duration_s":52.502163,"scheduling_latency_s":52.502178}1747gitlab # [ 244.609889] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:25.790Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"d469e3dae4d7e9b9abefd5ef","created_at":"2026-10-10T11:01:33.259Z","trace_propagation_headers":{"sentry-trace":"86c8ea7f1ae344faa892a74264cc672b-d5bda043f2c34112","baggage":"sentry-trace_id=86c8ea7f1ae344faa892a74264cc672b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQK9HZZ0DWXY545BJATFAW","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:33.262Z","job_size_bytes":89,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-d469e3dae4d7e9b9abefd5ef: done: 0.026008 sec","job_status":"done","queue_duration_s":52.502163,"scheduling_latency_s":52.502178,"redis_calls":11,"redis_duration_s":0.002457,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00076,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001286,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000411,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000282438,"duration_s":0.026008,"completed_at":"2026-10-10T11:02:25.790Z","load_balancing_strategy":"primary","db_duration_s":0.005208,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1748gitlab # [ 244.649187] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:25.791Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 fa8a9f48328d3d85c52fff7086159d5273f9ff52 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"fd329184b9108418268c4c4e","created_at":"2026-10-10T11:01:40.090Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-16c7c776301a40ff","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:40.094Z","job_size_bytes":185,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-fd329184b9108418268c4c4e: start","job_status":"start","queue_duration_s":45.697135,"scheduling_latency_s":45.69715}1749gitlab # [ 244.664686] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:25.802Z","message":"changes: \"0000000000000000000000000000000000000000 fa8a9f48328d3d85c52fff7086159d5273f9ff52 refs/heads/master\\n\""}1750gitlab # [ 244.666424] gitaly[904]: time="2026-10-10T11:02:25.840Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:35.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:25.828" grpc.time_ms=11.969 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1751gitlab # [ 244.667146] gitaly[904]: time="2026-10-10T11:02:25.854Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T11:02:35.024" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:02:25.844" grpc.time_ms=9.776 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1752gitlab # [ 244.673796] gitaly[904]: time="2026-10-10T11:02:25.917Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-10T11:02:35.088" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:25.908" grpc.time_ms=8.331 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1753gitlab # [ 244.707330] gitaly[904]: time="2026-10-10T11:02:25.950Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3929 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:25.950" grpc.time_ms=0.314 pid=904 span.kind=server system=grpc1754gitlab # [ 244.740695] gitaly[904]: time="2026-10-10T11:02:25.984Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3931 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:25.983" grpc.time_ms=0.341 pid=904 span.kind=server system=grpc1755gitlab # [ 244.763705] gitaly[904]: time="2026-10-10T11:02:26.007Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B diskcache=9f6f77dc-0dff-442e-92ce-e2442aac71ff grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T11:02:35.104" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:02:25.925" pid=904 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1756gitlab # [ 244.773634] gitaly[904]: time="2026-10-10T11:02:26.017Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=55 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5045 command.oublock=8 command.real_time_ms=71 command.spawn_token_fork_ms=0 command.system_time_ms=32 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T11:02:35.104" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:02:25.925" grpc.time_ms=82.242 pid=904 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1757gitlab # [ 244.796473] gitaly[904]: time="2026-10-10T11:02:26.039Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-10T11:02:36.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:02:26.039" grpc.time_ms=0.301 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1758gitlab # [ 244.824607] gitaly[904]: time="2026-10-10T11:02:26.068Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=940 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-10T11:02:36.023" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:26.053" grpc.time_ms=14.098 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1759gitlab # [ 244.908919] gitaly[904]: time="2026-10-10T11:02:26.152Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=341 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-10-10T11:02:56.044" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:26.144" grpc.time_ms=7.734 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1760gitlab # [ 244.946846] gitaly[904]: time="2026-10-10T11:02:26.190Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-10T11:02:36.159" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:02:26.189" grpc.time_ms=0.416 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1761gitlab # [ 244.958278] gitaly[904]: time="2026-10-10T11:02:26.201Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=340 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-10T11:02:36.163" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-10-10T11:02:26.194" grpc.time_ms=7.313 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1762gitlab # [ 245.004691] gitaly[904]: time="2026-10-10T11:02:26.248Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:36.207" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:26.237" grpc.time_ms=10.261 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1763gitlab # [ 245.023419] gitaly[904]: time="2026-10-10T11:02:26.266Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:36.225" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:26.255" grpc.time_ms=10.858 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1764gitlab # [ 245.055259] gitaly[904]: time="2026-10-10T11:02:26.298Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-10T11:02:56.181" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:26.282" grpc.time_ms=16.123 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1765gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b'1766gitlab # [ 245.416987] gitaly[904]: time="2026-10-10T11:02:26.660Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-10T11:02:36.008" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:26.659" grpc.time_ms=1.115 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1767gitlab # [ 245.458301] nixbot[2659]: {"ts": "2026-10-10T11:02:26.700Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35554 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b HTTP/1.1\" 200"}1768gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b', in 0.04 seconds)1769gitlab # [ 245.473260] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:26.715Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 fa8a9f48328d3d85c52fff7086159d5273f9ff52 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"fd329184b9108418268c4c4e","created_at":"2026-10-10T11:01:40.090Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-16c7c776301a40ff","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:01:40.094Z","job_size_bytes":185,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-fd329184b9108418268c4c4e: done: 0.924004 sec","job_status":"done","queue_duration_s":45.697135,"scheduling_latency_s":45.69715,"gitaly_calls":13,"gitaly_duration_s":0.263868,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.061499,"redis_read_bytes":2103,"redis_write_bytes":21748,"redis_cache_calls":15,"redis_cache_duration_s":0.005369,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.006823,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001605,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7836,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.006075,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.029724,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.011903,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.171,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.196,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.052,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.535599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000429384,"exclusive_lock_hold_duration_s":0.031345045,"duration_s":0.924004,"completed_at":"2026-10-10T11:02:26.715Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.093764,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1770gitlab # [ 245.521663] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:26.764Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"8ece366c7ae87eb4fa44a0fc","created_at":"2026-10-10T11:01:45.020Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-ac7b9777b00a4c3f","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:45.029Z","job_size_bytes":65,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-8ece366c7ae87eb4fa44a0fc: start","job_status":"start","queue_duration_s":41.735053,"scheduling_latency_s":41.735106}1771gitlab # [ 245.642934] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:26.885Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"8ece366c7ae87eb4fa44a0fc","created_at":"2026-10-10T11:01:45.020Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-ac7b9777b00a4c3f","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:45.029Z","job_size_bytes":65,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-8ece366c7ae87eb4fa44a0fc: done: 0.120727 sec","job_status":"done","queue_duration_s":41.735053,"scheduling_latency_s":41.735106,"redis_calls":16,"redis_duration_s":0.004291,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002178,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001616,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000497,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000473244,"duration_s":0.120727,"completed_at":"2026-10-10T11:02:26.885Z","load_balancing_strategy":"primary","db_duration_s":0.023578,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1772gitlab # [ 245.646210] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:26.886Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"79d54850515ba352979c5e30","created_at":"2026-10-10T11:01:45.030Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-ac7b9777b00a4c3f","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:45.035Z","job_size_bytes":65,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-79d54850515ba352979c5e30: start","job_status":"start","queue_duration_s":41.851462,"scheduling_latency_s":41.851479}1773gitlab # [ 245.905418] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:27.148Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"79d54850515ba352979c5e30","created_at":"2026-10-10T11:01:45.030Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-ac7b9777b00a4c3f","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:45.035Z","job_size_bytes":65,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-79d54850515ba352979c5e30: done: 0.261546 sec","job_status":"done","queue_duration_s":41.851462,"scheduling_latency_s":41.851479,"redis_calls":21,"redis_duration_s":0.005921,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002711,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001473,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001737,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.13368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000290819,"duration_s":0.261546,"completed_at":"2026-10-10T11:02:27.148Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034971,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1774gitlab # [ 245.908810] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:27.150Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"39bd7fc79a7bdbd805b3563b","created_at":"2026-10-10T11:01:45.135Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-ac7b9777b00a4c3f","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:45.140Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-39bd7fc79a7bdbd805b3563b: start","job_status":"start","queue_duration_s":42.009762,"scheduling_latency_s":42.009778}1775gitlab # [ 246.160993] gitaly[904]: time="2026-10-10T11:02:27.403Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:27.402" grpc.time_ms=0.608 pid=904 span.kind=server system=grpc1776gitlab # [ 246.216915] gitaly[904]: time="2026-10-10T11:02:27.459Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3888 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:27.458" grpc.time_ms=0.565 pid=904 span.kind=server system=grpc1777gitlab # [ 246.264445] gitaly[904]: time="2026-10-10T11:02:27.507Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:27.507" grpc.time_ms=0.618 pid=904 span.kind=server system=grpc1778gitlab # [ 246.286349] gitaly[904]: time="2026-10-10T11:02:27.529Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS diskcache=73f75b79-7993-46b5-bdfc-ab62c51382d5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T11:02:57.260" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:27.361" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1779gitlab # [ 246.295561] gitaly[904]: time="2026-10-10T11:02:27.539Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=162 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7550 command.oublock=0 command.real_time_ms=158 command.spawn_token_fork_ms=0 command.system_time_ms=106 command.user_time_ms=56 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T11:02:57.260" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:27.361" grpc.time_ms=168.459 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1780gitlab # [ 246.417634] gitaly[904]: time="2026-10-10T11:02:27.660Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T11:02:57.559" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:27.659" grpc.time_ms=1.331 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1781gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b'1782gitlab # [ 246.445620] gitaly[904]: time="2026-10-10T11:02:27.689Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKP0G3W4ZPQFAJRDJW8KS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:02:57.587" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:27.688" grpc.time_ms=0.797 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1783gitlab # [ 246.478490] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:27.721Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"39bd7fc79a7bdbd805b3563b","created_at":"2026-10-10T11:01:45.135Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-ac7b9777b00a4c3f","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:45.140Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2533,"job_status":"rescheduled","message":"PipelineProcessWorker JID-39bd7fc79a7bdbd805b3563b: rescheduled"}1784gitlab # [ 246.485434] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:27.728Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"39bd7fc79a7bdbd805b3563b","created_at":"2026-10-10T11:01:45.135Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-ac7b9777b00a4c3f","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T11:01:45.140Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-39bd7fc79a7bdbd805b3563b: done: 0.578788 sec","job_status":"done","queue_duration_s":42.009762,"scheduling_latency_s":42.009778,"gitaly_calls":3,"gitaly_duration_s":0.201806,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.040866,"redis_read_bytes":460,"redis_write_bytes":29288,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001579,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00111,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.022263,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.004313,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15575,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.005761,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000781,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000777,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002703,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.127,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.027,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.032,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.096,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.207898,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.468544186,"duration_s":0.578788,"completed_at":"2026-10-10T11:02:27.728Z","load_balancing_strategy":"primary","db_duration_s":0.140049,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1785gitlab # [ 246.503082] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:27.729Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","fa8a9f48328d3d85c52fff7086159d5273f9ff52 8827c58e7a9316066e7c9f4659b8c5b467dfaf9b refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"d523d3a8e5a1f7b73b8198ed","created_at":"2026-10-10T11:02:24.416Z","trace_propagation_headers":{"sentry-trace":"a89c908cbf1f4fc8953b71d574b0763a-7ef1a165c28a41d8","baggage":"sentry-trace_id=a89c908cbf1f4fc8953b71d574b0763a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQMWAZCZYVQCVA8QDZJF8C","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:02:24.420Z","job_size_bytes":185,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-d523d3a8e5a1f7b73b8198ed: start","job_status":"start","queue_duration_s":3.309852,"scheduling_latency_s":3.309867}1786gitlab # [ 246.546901] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:27.737Z","message":"changes: \"fa8a9f48328d3d85c52fff7086159d5273f9ff52 8827c58e7a9316066e7c9f4659b8c5b467dfaf9b refs/heads/master\\n\""}1787gitlab # [ 246.547652] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:27.774Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"373516f7f6737cbd0af4083b","created_at":"2026-10-10T11:02:27.772Z","trace_propagation_headers":{"sentry-trace":"a89c908cbf1f4fc8953b71d574b0763a-bd459aec2de84f9a","baggage":"sentry-trace_id=a89c908cbf1f4fc8953b71d574b0763a,sentry-release=Unknown"},"correlation_id":"01M4JQMWAZCZYVQCVA8QDZJF8C","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"8677d21a6ae2781ff1e487db","job_size_bytes":46,"pid":2533,"job_status":"deduplicated","message":"ProjectCacheWorker JID-373516f7f6737cbd0af4083b: deduplicated: until executing","deduplication.type":"until executing"}1788gitlab # [ 246.548660] gitaly[904]: time="2026-10-10T11:02:27.759Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:37.099" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:27.750" grpc.time_ms=9.023 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1789gitlab # [ 246.549655] gitaly[904]: time="2026-10-10T11:02:27.770Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-10T11:02:37.112" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:02:27.763" grpc.time_ms=7.661 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1790gitlab # [ 246.591313] gitaly[904]: time="2026-10-10T11:02:27.793Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-10T11:02:37.133" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:27.783" grpc.time_ms=9.174 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1791gitlab # [ 246.592187] gitaly[904]: time="2026-10-10T11:02:27.807Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-10-10T11:02:57.695" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:27.796" grpc.time_ms=10.849 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1792gitlab # [ 246.592918] gitaly[904]: time="2026-10-10T11:02:27.817Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=345 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-10T11:02:37.163" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-10-10T11:02:27.814" grpc.time_ms=3.835 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1793gitlab # [ 246.594110] gitaly[904]: time="2026-10-10T11:02:27.830Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:37.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:27.823" grpc.time_ms=6.793 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1794gitlab # [ 246.656965] nixbot[2659]: {"ts": "2026-10-10T11:02:27.900Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35556 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b HTTP/1.1\" 200"}1795gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b', in 0.20 seconds)1796gitlab # [ 246.669245] gitaly[904]: time="2026-10-10T11:02:27.912Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-10T11:02:37.091" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:27.911" grpc.time_ms=1.273 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1797gitlab # [ 246.683823] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:27.926Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","fa8a9f48328d3d85c52fff7086159d5273f9ff52 8827c58e7a9316066e7c9f4659b8c5b467dfaf9b refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"d523d3a8e5a1f7b73b8198ed","created_at":"2026-10-10T11:02:24.416Z","trace_propagation_headers":{"sentry-trace":"a89c908cbf1f4fc8953b71d574b0763a-7ef1a165c28a41d8","baggage":"sentry-trace_id=a89c908cbf1f4fc8953b71d574b0763a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQMWAZCZYVQCVA8QDZJF8C","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-10T11:02:24.420Z","job_size_bytes":185,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-d523d3a8e5a1f7b73b8198ed: done: 0.196873 sec","job_status":"done","queue_duration_s":3.309852,"scheduling_latency_s":3.309867,"gitaly_calls":7,"gitaly_duration_s":0.062672,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.018392000000000002,"redis_read_bytes":501,"redis_write_bytes":11942,"redis_cache_calls":9,"redis_cache_duration_s":0.00176,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001258,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5719,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003364,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.006237,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.005773,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.02,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.02,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.10624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000458718,"exclusive_lock_hold_duration_s":0.0162895,"duration_s":0.196873,"completed_at":"2026-10-10T11:02:27.926Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020212,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1798gitlab # [ 246.727622] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:27.970Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"8677d21a6ae2781ff1e487db","created_at":"2026-10-10T11:02:25.858Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-3b003c8411f7455d","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:25.882Z","job_size_bytes":46,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8677d21a6ae2781ff1e487db: start","job_status":"start","queue_duration_s":2.08875,"scheduling_latency_s":2.08879}1799gitlab # [ 246.797983] gitaly[904]: time="2026-10-10T11:02:28.041Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-10-10T17:02:28.035" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:02:28.036" grpc.time_ms=4.903 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1800gitlab # [ 246.817555] gitaly[904]: time="2026-10-10T11:02:28.061Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=654 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-10T11:02:58.052" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:28.053" grpc.time_ms=7.637 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1801gitlab # [ 246.852634] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:28.095Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"8677d21a6ae2781ff1e487db","created_at":"2026-10-10T11:02:25.858Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-3b003c8411f7455d","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:25.882Z","job_size_bytes":46,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8677d21a6ae2781ff1e487db: done: 0.124785 sec","job_status":"done","queue_duration_s":2.08875,"scheduling_latency_s":2.08879,"gitaly_calls":2,"gitaly_duration_s":0.018409,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008798,"redis_read_bytes":69,"redis_write_bytes":4879,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001586,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000784,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2112,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001794,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.0039,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":748,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000734,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068089,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000467378,"duration_s":0.124785,"completed_at":"2026-10-10T11:02:28.095Z","load_balancing_strategy":"primary","db_duration_s":0.015441,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1802gitlab # [ 246.856462] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:28.097Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"f455ab34deff64c7b36d86f4","created_at":"2026-10-10T11:02:25.899Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-3b003c8411f7455d","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:25.904Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-f455ab34deff64c7b36d86f4: start","job_status":"start","queue_duration_s":2.193534,"scheduling_latency_s":2.193574}1803gitlab # [ 246.926087] gitaly[904]: time="2026-10-10T11:02:28.169Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B diskcache=0c3b5c24-507a-4b48-8223-2a0c566916c8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-10T17:02:28.116" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:28.117" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1804gitlab # [ 246.927378] gitaly[904]: time="2026-10-10T11:02:28.169Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1288 command.oublock=0 command.real_time_ms=52 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-10T17:02:28.116" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:28.117" grpc.time_ms=52.41 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1805gitlab # [ 246.977668] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:28.220Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"f455ab34deff64c7b36d86f4","created_at":"2026-10-10T11:02:25.899Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-3b003c8411f7455d","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:25.904Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-f455ab34deff64c7b36d86f4: done: 0.123103 sec","job_status":"done","queue_duration_s":2.193534,"scheduling_latency_s":2.193574,"gitaly_calls":1,"gitaly_duration_s":0.055249,"redis_calls":15,"redis_duration_s":0.0040609999999999995,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0005,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001586,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000649,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001326,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.013,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.025,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041927,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.099600647,"duration_s":0.123103,"completed_at":"2026-10-10T11:02:28.220Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018769,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1806gitlab # [ 246.980378] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:28.222Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"2df9cd50742923136502880a","created_at":"2026-10-10T11:02:26.034Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-3b003c8411f7455d","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:26.036Z","job_size_bytes":23,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2df9cd50742923136502880a: start","job_status":"start","queue_duration_s":2.186213,"scheduling_latency_s":2.18624}1807gitlab # [ 247.031208] gitaly[904]: time="2026-10-10T11:02:28.260Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-10T11:02:58.248" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:28.249" grpc.time_ms=11.117 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1808gitlab # [ 247.053524] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:28.296Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"2df9cd50742923136502880a","created_at":"2026-10-10T11:02:26.034Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-3b003c8411f7455d","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:26.036Z","job_size_bytes":23,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-2df9cd50742923136502880a: done: 0.074193 sec","job_status":"done","queue_duration_s":2.186213,"scheduling_latency_s":2.18624,"gitaly_calls":1,"gitaly_duration_s":0.041745,"redis_calls":16,"redis_duration_s":0.004214,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000277,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001309,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001807,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000821,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023029,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.074193,"completed_at":"2026-10-10T11:02:28.296Z","load_balancing_strategy":"primary","db_duration_s":0.003465,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1809gitlab # [ 247.055324] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:28.298Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"54c665f03dd41da5eebfd59f","created_at":"2026-10-10T11:02:26.517Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-3b003c8411f7455d","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fbf5e1514402a24fbed4894f2b457a44ccc6b1fdcb60c9427217e6789e898067","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:26.524Z","job_size_bytes":467,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-54c665f03dd41da5eebfd59f: start","job_status":"start","queue_duration_s":1.773339,"scheduling_latency_s":1.77336}1810gitlab # [ 247.075727] gitaly[904]: time="2026-10-10T11:02:28.319Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:02:58.317" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:28.318" grpc.time_ms=0.878 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1811gitlab # [ 247.133585] gitaly[904]: time="2026-10-10T11:02:28.377Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-10T11:02:38.005" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:28.366" grpc.time_ms=10.458 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1812gitlab # [ 247.215622] gitaly[904]: time="2026-10-10T11:02:28.458Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T11:02:58.456" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:28.457" grpc.time_ms=1.217 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1813gitlab # [ 247.220718] gitaly[904]: time="2026-10-10T11:02:28.464Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=25 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-10T11:02:58.460" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:28.461" grpc.time_ms=2.752 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1814gitlab # [ 247.333340] gitaly[904]: time="2026-10-10T11:02:28.576Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:38.203" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:28.564" grpc.time_ms=12.264 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1815gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b'1816gitlab # [ 247.704850] nixbot[2659]: {"ts": "2026-10-10T11:02:28.947Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35564 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b HTTP/1.1\" 200"}1817gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b', in 0.05 seconds)1818gitlab # [ 247.739352] gitaly[904]: time="2026-10-10T11:02:28.982Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=961 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-10T11:02:58.867" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:28.968" grpc.time_ms=14.542 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1819gitlab # [ 247.761920] gitaly[904]: time="2026-10-10T11:02:28.999Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JQKGXX0NWJTYQ34B4RA28B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-10T11:02:58.886" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:28.987" grpc.time_ms=12.55 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1820gitlab # [ 247.792035] sidekiq-cluster[2533]: {"severity":"WARN","time":"2026-10-10T11:02:29.035Z","class":"CreatePipelineWorker","correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"fa8a9f48328d3d85c52fff7086159d5273f9ff52","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"fa8a9f48328d3d85c52fff7086159d5273f9ff52","ref":"refs/heads/master"},"retry":0}1821gitlab # [ 247.800446] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.043Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"54c665f03dd41da5eebfd59f","created_at":"2026-10-10T11:02:26.517Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-3b003c8411f7455d","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fbf5e1514402a24fbed4894f2b457a44ccc6b1fdcb60c9427217e6789e898067","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:26.524Z","job_size_bytes":467,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-54c665f03dd41da5eebfd59f: done: 0.74518 sec","job_status":"done","queue_duration_s":1.773339,"scheduling_latency_s":1.77336,"gitaly_calls":7,"gitaly_duration_s":0.159566,"redis_calls":51,"redis_duration_s":0.011442,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.001033,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004719,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001517,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003104,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.516036,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000481067,"duration_s":0.74518,"completed_at":"2026-10-10T11:02:29.043Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.051314,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1822gitlab # [ 247.843541] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.045Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"6fc8b737-9b9d-49b6-93ce-155b81ef4238\"}"],"class":"WebHookWorker","jid":"3ac30207fcd66ce826b00846","created_at":"2026-10-10T11:02:26.635Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-3b003c8411f7455d","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:846bde4431ce20fe79338c0bbe602dd0c2290b50e58c1c9116d577055a13ac08","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:26.639Z","job_size_bytes":1751,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-3ac30207fcd66ce826b00846: start","job_status":"start","queue_duration_s":2.405653,"scheduling_latency_s":2.40568}1823gitlab # [ 247.936438] nixbot[2659]: {"ts": "2026-10-10T11:02:29.179Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "49c0c2e2-5d36-43cd-835a-76d85c1d0b67", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "fa8a9f48328d3d85c52fff7086159d5273f9ff52", "pr": null}1824gitlab # [ 247.946172] nixbot[2659]: {"ts": "2026-10-10T11:02:29.189Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35574 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}1825gitlab # [ 247.971538] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.214Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"6fc8b737-9b9d-49b6-93ce-155b81ef4238\"}"],"class":"WebHookWorker","jid":"3ac30207fcd66ce826b00846","created_at":"2026-10-10T11:02:26.635Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-3b003c8411f7455d","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:846bde4431ce20fe79338c0bbe602dd0c2290b50e58c1c9116d577055a13ac08","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:26.639Z","job_size_bytes":1751,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-3ac30207fcd66ce826b00846: done: 0.168576 sec","job_status":"done","queue_duration_s":2.405653,"scheduling_latency_s":2.40568,"redis_calls":19,"redis_duration_s":0.004589,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001343,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.00049,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001875,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000881,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.0663372019999997,"cpu_s":0.0861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049308,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.168576,"completed_at":"2026-10-10T11:02:29.214Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003141,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1826gitlab # [ 248.035319] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.223Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"14f357aa4c9465b236ae37fe","created_at":"2026-10-10T11:02:27.343Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.348Z","job_size_bytes":23,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-14f357aa4c9465b236ae37fe: start","job_status":"start","queue_duration_s":1.874161,"scheduling_latency_s":1.874204}1827gitlab # [ 248.061416] gitaly[904]: time="2026-10-10T11:02:29.304Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=35 command.spawn_token_fork_ms=27 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JQN1K928XXR48VM198S6BV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:29.268" grpc.time_ms=36.375 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1828gitlab # [ 248.063246] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:29 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 791829gitlab # [ 248.115305] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.358Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"14f357aa4c9465b236ae37fe","created_at":"2026-10-10T11:02:27.343Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.348Z","job_size_bytes":23,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-14f357aa4c9465b236ae37fe: done: 0.135421 sec","job_status":"done","queue_duration_s":1.874161,"scheduling_latency_s":1.874204,"redis_calls":10,"redis_duration_s":0.002282,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000576,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001209,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000497,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.06,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.057,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000210641,"duration_s":0.135421,"completed_at":"2026-10-10T11:02:29.358Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.064314,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1830gitlab # [ 248.116986] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.360Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"15acf7df32b820d29ab5c8da","created_at":"2026-10-10T11:02:27.536Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.557Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-15acf7df32b820d29ab5c8da: start","job_status":"start","queue_duration_s":1.802885,"scheduling_latency_s":1.8029}1831gitlab # [ 248.155832] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.382Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"15acf7df32b820d29ab5c8da","created_at":"2026-10-10T11:02:27.536Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.557Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-15acf7df32b820d29ab5c8da: done: 0.021576 sec","job_status":"done","queue_duration_s":1.802885,"scheduling_latency_s":1.8029,"redis_calls":10,"redis_duration_s":0.001286,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000265,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000762,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021595,"duration_s":0.021576,"completed_at":"2026-10-10T11:02:29.381Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004006,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1832gitlab # [ 248.202045] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.382Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"3c84e0983ca5de14e4449311","created_at":"2026-10-10T11:02:27.573Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.582Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-3c84e0983ca5de14e4449311: start","job_status":"start","queue_duration_s":1.800103,"scheduling_latency_s":1.800117}1833gitlab # [ 248.202739] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.409Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"3c84e0983ca5de14e4449311","created_at":"2026-10-10T11:02:27.573Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.582Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-3c84e0983ca5de14e4449311: done: 0.02649 sec","job_status":"done","queue_duration_s":1.800103,"scheduling_latency_s":1.800117,"redis_calls":10,"redis_duration_s":0.004679,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000239,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003543,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000897,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000893968,"duration_s":0.02649,"completed_at":"2026-10-10T11:02:29.409Z","load_balancing_strategy":"primary","db_duration_s":0.005608,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1834gitlab # [ 248.219523] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.410Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"0b12495f7c88b2961e80eaeb","created_at":"2026-10-10T11:02:27.600Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.605Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-0b12495f7c88b2961e80eaeb: start","job_status":"start","queue_duration_s":1.804548,"scheduling_latency_s":1.804561}1835gitlab # [ 248.233210] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.435Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"0b12495f7c88b2961e80eaeb","created_at":"2026-10-10T11:02:27.600Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.605Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-0b12495f7c88b2961e80eaeb: done: 0.024769 sec","job_status":"done","queue_duration_s":1.804548,"scheduling_latency_s":1.804561,"redis_calls":13,"redis_duration_s":0.001607,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000235,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000751,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000377,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000244,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000197232,"duration_s":0.024769,"completed_at":"2026-10-10T11:02:29.435Z","load_balancing_strategy":"primary","db_duration_s":0.00544,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1836gitlab # [ 248.276671] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.436Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e0a82fb84f562b9414f7a26f","created_at":"2026-10-10T11:02:27.606Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.611Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e0a82fb84f562b9414f7a26f: start","job_status":"start","queue_duration_s":1.824898,"scheduling_latency_s":1.824912}1837gitlab # [ 248.287094] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.462Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e0a82fb84f562b9414f7a26f","created_at":"2026-10-10T11:02:27.606Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.611Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e0a82fb84f562b9414f7a26f: done: 0.026234 sec","job_status":"done","queue_duration_s":1.824898,"scheduling_latency_s":1.824912,"redis_calls":10,"redis_duration_s":0.005822,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000261,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004763,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000798,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001068851,"duration_s":0.026234,"completed_at":"2026-10-10T11:02:29.462Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004599,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1838gitlab # [ 248.330677] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.464Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"a7a0115f018bcce8c061d4ed","created_at":"2026-10-10T11:02:27.611Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.617Z","job_size_bytes":19,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-a7a0115f018bcce8c061d4ed: start","job_status":"start","queue_duration_s":1.846224,"scheduling_latency_s":1.846239}1839gitlab # [ 248.339844] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.487Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"a7a0115f018bcce8c061d4ed","created_at":"2026-10-10T11:02:27.611Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.617Z","job_size_bytes":19,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-a7a0115f018bcce8c061d4ed: done: 0.023074 sec","job_status":"done","queue_duration_s":1.846224,"scheduling_latency_s":1.846239,"redis_calls":11,"redis_duration_s":0.003221,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000857,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001931,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000433,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000195835,"duration_s":0.023074,"completed_at":"2026-10-10T11:02:29.487Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004912,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1840gitlab # [ 248.382678] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.488Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"fcc9ee6a80ba8ac8ca35274c","created_at":"2026-10-10T11:02:27.620Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.623Z","job_size_bytes":66,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-fcc9ee6a80ba8ac8ca35274c: start","job_status":"start","queue_duration_s":1.864426,"scheduling_latency_s":1.86444}1841gitlab # [ 248.393613] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.538Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"fcc9ee6a80ba8ac8ca35274c","created_at":"2026-10-10T11:02:27.620Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.623Z","job_size_bytes":66,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-fcc9ee6a80ba8ac8ca35274c: done: 0.05051 sec","job_status":"done","queue_duration_s":1.864426,"scheduling_latency_s":1.86444,"redis_calls":10,"redis_duration_s":0.005489,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00048,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004286,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000723,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000898997,"duration_s":0.05051,"completed_at":"2026-10-10T11:02:29.538Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028699,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1842gitlab # [ 248.442721] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.539Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"24c00cc23c1fd84bf0c0e0ce","created_at":"2026-10-10T11:02:27.624Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.627Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-24c00cc23c1fd84bf0c0e0ce: start","job_status":"start","queue_duration_s":1.912365,"scheduling_latency_s":1.912379}1843gitlab # [ 248.446814] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.566Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"24c00cc23c1fd84bf0c0e0ce","created_at":"2026-10-10T11:02:27.624Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.627Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-24c00cc23c1fd84bf0c0e0ce: done: 0.026563 sec","job_status":"done","queue_duration_s":1.912365,"scheduling_latency_s":1.912379,"redis_calls":10,"redis_duration_s":0.001266,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000263,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000215111,"duration_s":0.026563,"completed_at":"2026-10-10T11:02:29.566Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005334,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1844gitlab # [ 248.485482] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.567Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"50cc74b872e9e36a80daa3ca","created_at":"2026-10-10T11:02:27.632Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.635Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-50cc74b872e9e36a80daa3ca: start","job_status":"start","queue_duration_s":1.932011,"scheduling_latency_s":1.932025}1845gitlab # [ 248.499775] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.597Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"50cc74b872e9e36a80daa3ca","created_at":"2026-10-10T11:02:27.632Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.635Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-50cc74b872e9e36a80daa3ca: done: 0.029878 sec","job_status":"done","queue_duration_s":1.932011,"scheduling_latency_s":1.932025,"redis_calls":10,"redis_duration_s":0.004509,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003323,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205613,"duration_s":0.029878,"completed_at":"2026-10-10T11:02:29.597Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01005,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1846gitlab # [ 248.550204] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.598Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e03681d47ad26c2e93d69399","created_at":"2026-10-10T11:02:27.635Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.638Z","job_size_bytes":6,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e03681d47ad26c2e93d69399: start","job_status":"start","queue_duration_s":1.95961,"scheduling_latency_s":1.959623}1847gitlab # [ 248.552106] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.613Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e03681d47ad26c2e93d69399","created_at":"2026-10-10T11:02:27.635Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.638Z","job_size_bytes":6,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e03681d47ad26c2e93d69399: done: 0.015019 sec","job_status":"done","queue_duration_s":1.95961,"scheduling_latency_s":1.959623,"redis_calls":10,"redis_duration_s":0.001291,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000259,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000774,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000258,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00021539,"duration_s":0.015019,"completed_at":"2026-10-10T11:02:29.613Z","load_balancing_strategy":"primary","db_duration_s":0.002334,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1848gitlab # [ 248.600653] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.614Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"c4a564cf22e94a577b536cb6","created_at":"2026-10-10T11:02:27.638Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.642Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-c4a564cf22e94a577b536cb6: start","job_status":"start","queue_duration_s":1.971728,"scheduling_latency_s":1.971741}1849gitlab # [ 248.604498] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.703Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"c4a564cf22e94a577b536cb6","created_at":"2026-10-10T11:02:27.638Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.642Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-c4a564cf22e94a577b536cb6: done: 0.089154 sec","job_status":"done","queue_duration_s":1.971728,"scheduling_latency_s":1.971741,"redis_calls":10,"redis_duration_s":0.00333,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000243,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000261485,"duration_s":0.089154,"completed_at":"2026-10-10T11:02:29.703Z","load_balancing_strategy":"primary","db_duration_s":0.006758,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1850gitlab # [ 248.662569] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.704Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"14611fd00269d4632039a932","created_at":"2026-10-10T11:02:27.646Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.649Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-14611fd00269d4632039a932: start","job_status":"start","queue_duration_s":2.054765,"scheduling_latency_s":2.054783}1851gitlab # [ 248.663767] gitaly[904]: time="2026-10-10T11:02:29.822Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C diskcache=c103d5e4-4f52-4da2-bfd5-a5c7e5368821 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-10T17:02:29.811" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:29.812" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1852gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b'1853gitlab # [ 248.664677] gitaly[904]: time="2026-10-10T11:02:29.822Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=312 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-10T17:02:29.811" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:29.812" grpc.time_ms=10.801 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1854gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b', in 0.03 seconds)1855gitlab # [ 248.708961] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.761Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"14611fd00269d4632039a932","created_at":"2026-10-10T11:02:27.646Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.649Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-14611fd00269d4632039a932: done: 0.057559 sec","job_status":"done","queue_duration_s":2.054765,"scheduling_latency_s":2.054783,"redis_calls":14,"redis_duration_s":0.005594999999999999,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002034,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003312,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000508165,"duration_s":0.057559,"completed_at":"2026-10-10T11:02:29.761Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014212,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1856gitlab # [ 248.714525] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.763Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"abbd19858245edbf85ccfff4","created_at":"2026-10-10T11:02:27.722Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.723Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-abbd19858245edbf85ccfff4: start","job_status":"start","queue_duration_s":2.040491,"scheduling_latency_s":2.040523}1857gitlab # [ 248.759293] gitaly[904]: time="2026-10-10T11:02:29.847Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:02:59.846" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:29.846" grpc.time_ms=0.509 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1858gitlab # [ 248.760171] nixbot[2659]: {"ts": "2026-10-10T11:02:29.990Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35590 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b HTTP/1.1\" 200"}1859gitlab # [ 248.760675] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.796Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"abbd19858245edbf85ccfff4","created_at":"2026-10-10T11:02:27.722Z","trace_propagation_headers":{"sentry-trace":"ec871099bc2e434cb7fd104a0796f8c0-89f289f9e46b45d5","baggage":"sentry-trace_id=ec871099bc2e434cb7fd104a0796f8c0,sentry-release=Unknown"},"correlation_id":"01M4JQKP0G3W4ZPQFAJRDJW8KS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.723Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-abbd19858245edbf85ccfff4: done: 0.032957 sec","job_status":"done","queue_duration_s":2.040491,"scheduling_latency_s":2.040523,"redis_calls":8,"redis_duration_s":0.00149,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001145,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000345,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032957,"completed_at":"2026-10-10T11:02:29.796Z","load_balancing_strategy":"primary","db_duration_s":0.00365,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1860gitlab # [ 248.813806] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.798Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8c33d861eb615849fad9b4b9","created_at":"2026-10-10T11:02:27.778Z","trace_propagation_headers":{"sentry-trace":"a89c908cbf1f4fc8953b71d574b0763a-bd459aec2de84f9a","baggage":"sentry-trace_id=a89c908cbf1f4fc8953b71d574b0763a,sentry-release=Unknown"},"correlation_id":"01M4JQMWAZCZYVQCVA8QDZJF8C","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"f455ab34deff64c7b36d86f4","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.780Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8c33d861eb615849fad9b4b9: start","job_status":"start","queue_duration_s":2.017884,"scheduling_latency_s":2.017911}1861gitlab # [ 248.814685] gitaly[904]: time="2026-10-10T11:02:29.854Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-10T11:02:39.198" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:29.849" grpc.time_ms=5.067 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1862gitlab # [ 248.815492] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.832Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8c33d861eb615849fad9b4b9","created_at":"2026-10-10T11:02:27.778Z","trace_propagation_headers":{"sentry-trace":"a89c908cbf1f4fc8953b71d574b0763a-bd459aec2de84f9a","baggage":"sentry-trace_id=a89c908cbf1f4fc8953b71d574b0763a,sentry-release=Unknown"},"correlation_id":"01M4JQMWAZCZYVQCVA8QDZJF8C","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"f455ab34deff64c7b36d86f4","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.780Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8c33d861eb615849fad9b4b9: done: 0.034128 sec","job_status":"done","queue_duration_s":2.017884,"scheduling_latency_s":2.017911,"gitaly_calls":1,"gitaly_duration_s":0.012523,"redis_calls":12,"redis_duration_s":0.0016179999999999999,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000667,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000336,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000615,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.020700117,"duration_s":0.034128,"completed_at":"2026-10-10T11:02:29.832Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002646,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1863gitlab # [ 248.867071] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:29.833Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"dd203ac781ed1c6c1c09c4b8","created_at":"2026-10-10T11:02:27.882Z","trace_propagation_headers":{"sentry-trace":"a89c908cbf1f4fc8953b71d574b0763a-bd459aec2de84f9a","baggage":"sentry-trace_id=a89c908cbf1f4fc8953b71d574b0763a,sentry-release=Unknown"},"correlation_id":"01M4JQMWAZCZYVQCVA8QDZJF8C","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d74f8137f007e6aa703319dce3ce31e59f01f0802235936a4530e525db45f963","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.884Z","job_size_bytes":467,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-dd203ac781ed1c6c1c09c4b8: start","job_status":"start","queue_duration_s":1.949288,"scheduling_latency_s":1.949302}1864gitlab # [ 248.867707] gitaly[904]: time="2026-10-10T11:02:29.874Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T11:02:59.873" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:29.873" grpc.time_ms=0.532 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1865gitlab # [ 248.868668] gitaly[904]: time="2026-10-10T11:02:29.877Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=37 command.oublock=0 command.real_time_ms=0 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-10T11:02:59.876" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:29.876" grpc.time_ms=1.321 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1866gitlab # [ 248.907532] gitaly[904]: time="2026-10-10T11:02:29.936Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:39.280" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:29.931" grpc.time_ms=4.942 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1867gitlab # [ 248.935677] gitaly[904]: time="2026-10-10T11:02:30.179Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN1P3H243KDY97136JB71 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:29.352" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1868gitlab # [ 248.936589] gitaly[904]: time="2026-10-10T11:02:30.179Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=81 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=824 command.spawn_token_fork_ms=818 command.system_time_ms=79 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN1P3H243KDY97136JB71 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:29.352" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1869gitlab # [ 248.937210] gitaly[904]: time="2026-10-10T11:02:30.179Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=81 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=824 command.spawn_token_fork_ms=818 command.system_time_ms=79 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN1P3H243KDY97136JB71 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:29.352" grpc.time_ms=826.79 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1870gitlab # [ 248.969377] gitlab-workhorse[825]: time="2026-10-10T11:02:30Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671871gitlab # [ 248.970325] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:30 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 8971872gitlab # [ 248.999843] gitaly[904]: time="2026-10-10T11:02:30.243Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-10T11:03:00.231" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:30.231" grpc.time_ms=11.593 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1873gitlab # [ 249.013454] gitaly[904]: time="2026-10-10T11:02:30.255Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JQMWAZCZYVQCVA8QDZJF8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-10T11:03:00.247" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:30.247" grpc.time_ms=7.859 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1874gitlab # [ 249.031332] gitaly[904]: time="2026-10-10T11:02:30.274Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN2J84FRVJ3P9Z567TFJJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:30.265" pid=904 remote_ip="2001:db8:1::1" request_sha=c35a2bc2ce3e72c22fb3db9326e04f8f463ca88c response_bytes=337 span.kind=server system=grpc1875gitlab # [ 249.032132] gitaly[904]: time="2026-10-10T11:02:30.275Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1186 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN2J84FRVJ3P9Z567TFJJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:30.265" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1876gitlab # [ 249.032684] gitaly[904]: time="2026-10-10T11:02:30.275Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1186 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN2J84FRVJ3P9Z567TFJJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:30.265" grpc.time_ms=9.975 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1877gitlab # [ 249.039069] sidekiq-cluster[2533]: {"severity":"WARN","time":"2026-10-10T11:02:30.282Z","class":"CreatePipelineWorker","correlation_id":"01M4JQMWAZCZYVQCVA8QDZJF8C","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"fa8a9f48328d3d85c52fff7086159d5273f9ff52","after":"8827c58e7a9316066e7c9f4659b8c5b467dfaf9b","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"8827c58e7a9316066e7c9f4659b8c5b467dfaf9b","ref":"refs/heads/master"},"retry":0}1878gitlab # [ 249.045205] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:30.288Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"dd203ac781ed1c6c1c09c4b8","created_at":"2026-10-10T11:02:27.882Z","trace_propagation_headers":{"sentry-trace":"a89c908cbf1f4fc8953b71d574b0763a-bd459aec2de84f9a","baggage":"sentry-trace_id=a89c908cbf1f4fc8953b71d574b0763a,sentry-release=Unknown"},"correlation_id":"01M4JQMWAZCZYVQCVA8QDZJF8C","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d74f8137f007e6aa703319dce3ce31e59f01f0802235936a4530e525db45f963","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.884Z","job_size_bytes":467,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-dd203ac781ed1c6c1c09c4b8: done: 0.454912 sec","job_status":"done","queue_duration_s":1.949288,"scheduling_latency_s":1.949302,"gitaly_calls":7,"gitaly_duration_s":0.095112,"redis_calls":33,"redis_duration_s":0.0035369999999999998,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.00039,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000789,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.00154,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000818,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.383281,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.454912,"completed_at":"2026-10-10T11:02:30.288Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01165,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1879gitlab # [ 249.103816] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:30.290Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"ffb6f3fd-aebc-42c7-aec9-da706241b605\"}"],"class":"WebHookWorker","jid":"bfb7d728ca5059272bee3b56","created_at":"2026-10-10T11:02:27.896Z","trace_propagation_headers":{"sentry-trace":"a89c908cbf1f4fc8953b71d574b0763a-bd459aec2de84f9a","baggage":"sentry-trace_id=a89c908cbf1f4fc8953b71d574b0763a,sentry-release=Unknown"},"correlation_id":"01M4JQMWAZCZYVQCVA8QDZJF8C","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:869d1f6d43dbfb9179b5f6d933e0a3fd4cba4d878b1361cf516681f7130dd8df","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.900Z","job_size_bytes":1749,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-bfb7d728ca5059272bee3b56: start","job_status":"start","queue_duration_s":2.389927,"scheduling_latency_s":2.389958}1880gitlab # [ 249.104603] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:30.317Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"ffb6f3fd-aebc-42c7-aec9-da706241b605\"}"],"class":"WebHookWorker","jid":"bfb7d728ca5059272bee3b56","created_at":"2026-10-10T11:02:27.896Z","trace_propagation_headers":{"sentry-trace":"a89c908cbf1f4fc8953b71d574b0763a-bd459aec2de84f9a","baggage":"sentry-trace_id=a89c908cbf1f4fc8953b71d574b0763a,sentry-release=Unknown"},"correlation_id":"01M4JQMWAZCZYVQCVA8QDZJF8C","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:869d1f6d43dbfb9179b5f6d933e0a3fd4cba4d878b1361cf516681f7130dd8df","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:27.900Z","job_size_bytes":1749,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-bfb7d728ca5059272bee3b56: done: 0.027932 sec","job_status":"done","queue_duration_s":2.389927,"scheduling_latency_s":2.389958,"redis_calls":14,"redis_duration_s":0.001824,"redis_read_bytes":25,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000276,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000982,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000566,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.005044774999987567,"cpu_s":0.016488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.027932,"completed_at":"2026-10-10T11:02:30.317Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001392,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1881gitlab # [ 249.150104] nixbot[2659]: {"ts": "2026-10-10T11:02:30.307Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "1043f50d-6c5b-4d4e-8d62-06b0fc832bbc", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "8827c58e7a9316066e7c9f4659b8c5b467dfaf9b", "pr": null}1882gitlab # [ 249.150809] nixbot[2659]: {"ts": "2026-10-10T11:02:30.310Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35594 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}1883gitlab # [ 249.151489] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:30.319Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"80ab331c63335ad7fd0e0da8","created_at":"2026-10-10T11:02:28.075Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-b54fb77acd9f4b43","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:28.082Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-80ab331c63335ad7fd0e0da8: start","job_status":"start","queue_duration_s":2.236989,"scheduling_latency_s":2.237012}1884gitlab # [ 249.152177] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:30.351Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"80ab331c63335ad7fd0e0da8","created_at":"2026-10-10T11:02:28.075Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-b54fb77acd9f4b43","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:28.082Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-80ab331c63335ad7fd0e0da8: done: 0.032071 sec","job_status":"done","queue_duration_s":2.236989,"scheduling_latency_s":2.237012,"redis_calls":19,"redis_duration_s":0.002567,"redis_read_bytes":21,"redis_write_bytes":4290,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000489,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000311,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2261,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001299,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000468,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022312,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.004797816,"duration_s":0.032071,"completed_at":"2026-10-10T11:02:30.351Z","load_balancing_strategy":"primary","db_duration_s":0.004696,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1885gitlab # [ 249.199627] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:30.352Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","3ac30207fcd66ce826b00846"],"class":"WebHooks::LogExecutionWorker","jid":"69aa9fabc393db157b96c462","created_at":"2026-10-10T11:02:29.193Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-c2f78ca9928e431a","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:387a6d77d9587cbf3d3db8c79663c6d64e5a4e476f942365601a6ad330b41877","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:29.202Z","job_size_bytes":2593,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-69aa9fabc393db157b96c462: start","job_status":"start","queue_duration_s":1.149342,"scheduling_latency_s":1.149357}1886gitlab # [ 249.204494] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:30.381Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","3ac30207fcd66ce826b00846"],"class":"WebHooks::LogExecutionWorker","jid":"69aa9fabc393db157b96c462","created_at":"2026-10-10T11:02:29.193Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-c2f78ca9928e431a","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:387a6d77d9587cbf3d3db8c79663c6d64e5a4e476f942365601a6ad330b41877","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:29.202Z","job_size_bytes":2593,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-69aa9fabc393db157b96c462: done: 0.029676 sec","job_status":"done","queue_duration_s":1.149342,"scheduling_latency_s":1.149357,"redis_calls":10,"redis_duration_s":0.001259,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000235,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000253,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000210641,"duration_s":0.029676,"completed_at":"2026-10-10T11:02:30.381Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005179,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1887gitlab # [ 249.270502] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:30.382Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","bfb7d728ca5059272bee3b56"],"class":"WebHooks::LogExecutionWorker","jid":"741353921492239b66d36bb5","created_at":"2026-10-10T11:02:30.311Z","trace_propagation_headers":{"sentry-trace":"a89c908cbf1f4fc8953b71d574b0763a-5dd199dd46134d20","baggage":"sentry-trace_id=a89c908cbf1f4fc8953b71d574b0763a,sentry-release=Unknown"},"correlation_id":"01M4JQMWAZCZYVQCVA8QDZJF8C","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c471f28d864ce44bb95f6461053d086da22ccd1e7ab415e9ad7837d77f9d1ec8","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:30.312Z","job_size_bytes":2592,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-741353921492239b66d36bb5: start","job_status":"start","queue_duration_s":0.070562,"scheduling_latency_s":0.070578}1888gitlab # [ 249.271145] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:30.397Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","bfb7d728ca5059272bee3b56"],"class":"WebHooks::LogExecutionWorker","jid":"741353921492239b66d36bb5","created_at":"2026-10-10T11:02:30.311Z","trace_propagation_headers":{"sentry-trace":"a89c908cbf1f4fc8953b71d574b0763a-5dd199dd46134d20","baggage":"sentry-trace_id=a89c908cbf1f4fc8953b71d574b0763a,sentry-release=Unknown"},"correlation_id":"01M4JQMWAZCZYVQCVA8QDZJF8C","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c471f28d864ce44bb95f6461053d086da22ccd1e7ab415e9ad7837d77f9d1ec8","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:30.312Z","job_size_bytes":2592,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-741353921492239b66d36bb5: done: 0.014913 sec","job_status":"done","queue_duration_s":0.070562,"scheduling_latency_s":0.070578,"redis_calls":7,"redis_duration_s":0.000961,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000655,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014913,"completed_at":"2026-10-10T11:02:30.397Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002462,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1889gitlab # [ 249.295301] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:30.398Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"e7d5d75ffc869af8685ae2a6","created_at":"2026-10-10T11:02:30.341Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-9eef7ad7ad6642d9","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:30.344Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-e7d5d75ffc869af8685ae2a6: start","job_status":"start","queue_duration_s":0.054161,"scheduling_latency_s":0.054175}1890gitlab # [ 249.310098] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:30.453Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"e7d5d75ffc869af8685ae2a6","created_at":"2026-10-10T11:02:30.341Z","trace_propagation_headers":{"sentry-trace":"dc1dcfd703554658962c0f7f2ad179a0-9eef7ad7ad6642d9","baggage":"sentry-trace_id=dc1dcfd703554658962c0f7f2ad179a0,sentry-release=Unknown"},"correlation_id":"01M4JQKGXX0NWJTYQ34B4RA28B","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:30.344Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-e7d5d75ffc869af8685ae2a6: done: 0.054658 sec","job_status":"done","queue_duration_s":0.054161,"scheduling_latency_s":0.054175,"redis_calls":11,"redis_duration_s":0.001548,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001027,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000220698,"duration_s":0.054658,"completed_at":"2026-10-10T11:02:30.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012221,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1891gitlab # [ 249.388508] gitlab-workhorse[825]: time="2026-10-10T11:02:30Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3371892gitlab # [ 249.389253] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:30 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 4141893gitlab # [ 249.419271] nixbot[2659]: {"ts": "2026-10-10T11:02:30.662Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1894gitlab # [ 249.421273] nixbot[2659]: {"ts": "2026-10-10T11:02:30.664Z", "level": "info", "logger": "nixbot.after_build", "msg": "reused build: replaying effect statuses", "build_id": 1, "sha": "fa8a9f48328d3d85c52fff7086159d5273f9ff52"}1895gitlab # [ 249.464786] gitaly[904]: time="2026-10-10T11:02:30.708Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JQN308EQ8B3K32GRY9KKBX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:30.698" grpc.time_ms=9.38 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1896gitlab # [ 249.465831] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:30 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 441897gitlab # [ 249.479748] gitaly[904]: time="2026-10-10T11:02:30.723Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:00.222" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:30.722" grpc.time_ms=0.638 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1898gitlab # [ 249.506689] gitaly[904]: time="2026-10-10T11:02:30.750Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN31QGGC2NFRWTKYJPW69 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:30.744" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1899gitlab # [ 249.507512] gitaly[904]: time="2026-10-10T11:02:30.750Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN31QGGC2NFRWTKYJPW69 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:30.744" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1900gitlab # [ 249.508401] gitlab-workhorse[825]: time="2026-10-10T11:02:30Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671901gitlab # [ 249.508928] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:30 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 391902gitlab # [ 249.510641] gitaly[904]: time="2026-10-10T11:02:30.750Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN31QGGC2NFRWTKYJPW69 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:30.744" grpc.time_ms=6.018 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1903gitlab # [ 249.560529] gitaly[904]: time="2026-10-10T11:02:30.803Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:40.157" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:30.798" grpc.time_ms=5.632 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1904gitlab # [ 249.611092] gitaly[904]: time="2026-10-10T11:02:30.854Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQN351EYJ4V4N8AV1P4JXV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:30.849" grpc.time_ms=5.353 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1905gitlab # [ 249.612070] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:30 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 371906gitlab # [ 249.635057] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:30.878Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"09e5490aa8ae07939ceff373","created_at":"2026-10-10T11:02:30.873Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-9cb4100418944d4d","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:30.876Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-09e5490aa8ae07939ceff373: start","job_status":"start","queue_duration_s":0.001413,"scheduling_latency_s":0.001448}1907gitlab # [ 249.645970] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fa8a9f48328d3d85c52fff7086159d5273f9ff52 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 2151908gitlab # [ 249.656731] gitaly[904]: time="2026-10-10T11:02:30.899Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN369ZT4A3MNZ6DM7NKWD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:30.887" pid=904 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc1909gitlab # [ 249.666423] gitaly[904]: time="2026-10-10T11:02:30.900Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1172 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN369ZT4A3MNZ6DM7NKWD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:30.887" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1910gitlab # [ 249.679616] gitlab-workhorse[825]: time="2026-10-10T11:02:30Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2161911gitlab # [ 249.681449] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:30 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 431912gitlab # [ 249.683272] gitaly[904]: time="2026-10-10T11:02:30.900Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1172 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN369ZT4A3MNZ6DM7NKWD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:30.887" grpc.time_ms=12.598 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1913gitlab # [ 249.688757] gitaly[904]: time="2026-10-10T11:02:30.932Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN37CFWT2TXM0FFTH0QKR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:00.431" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:30.931" grpc.time_ms=0.357 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1914gitlab # [ 249.718933] gitaly[904]: time="2026-10-10T11:02:30.962Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4JQN37CFWT2TXM0FFTH0QKR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:40.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:30.957" grpc.time_ms=5.312 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1915gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b'1916gitlab # [ 249.751365] gitaly[904]: time="2026-10-10T11:02:30.994Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:30.994" grpc.time_ms=0.298 pid=904 span.kind=server system=grpc1917gitlab # [ 249.760665] gitaly[904]: time="2026-10-10T11:02:31.003Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN39J2YRXFMH163Q544R8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:00.399" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:30.999" grpc.time_ms=3.887 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1918gitlab # [ 249.772729] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fa8a9f48328d3d85c52fff7086159d5273f9ff52 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1091919gitlab # [ 249.778531] gitaly[904]: time="2026-10-10T11:02:31.022Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQN39J2YRXFMH163Q544R8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-10T11:03:01.010" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:31.011" grpc.time_ms=10.813 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1920gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b', in 0.04 seconds)1921(finished: subtest: push delivers the webhook, in 46.88 seconds)1922subtest: eval and build post commit statuses1923gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses1924gitlab # [ 249.790966] gitaly[904]: time="2026-10-10T11:02:31.024Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3888 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:31.023" grpc.time_ms=0.8 pid=904 span.kind=server system=grpc1925gitlab # [ 249.802143] nixbot[2659]: {"ts": "2026-10-10T11:02:31.032Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:35596 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b HTTP/1.1\" 200"}1926gitlab # [ 249.804710] gitaly[904]: time="2026-10-10T11:02:31.045Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQN39J2YRXFMH163Q544R8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:41.014" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:31.034" grpc.time_ms=10.677 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1927gitlab # [ 249.816705] gitaly[904]: time="2026-10-10T11:02:31.058Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:31.058" grpc.time_ms=0.68 pid=904 span.kind=server system=grpc1928gitlab # [ 249.825836] gitaly[904]: time="2026-10-10T11:02:31.062Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN3B0RVPDEXYT1T54TNEM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:01.061" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:31.061" grpc.time_ms=0.302 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1929gitlab # [ 249.837205] gitaly[904]: time="2026-10-10T11:02:31.070Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 diskcache=88a7f6f8-8118-41d2-9973-1d4bd31aa1f2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T11:03:00.869" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:30.970" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1930gitlab # [ 249.846386] gitaly[904]: time="2026-10-10T11:02:31.070Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=68 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7205 command.oublock=0 command.real_time_ms=92 command.spawn_token_fork_ms=4 command.system_time_ms=46 command.user_time_ms=22 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T11:03:00.869" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:30.970" grpc.time_ms=100.435 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1931gitlab # [ 249.859322] gitaly[904]: time="2026-10-10T11:02:31.072Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN39J2YRXFMH163Q544R8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-10T11:02:41.052" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:02:31.072" grpc.time_ms=0.286 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1932gitlab # [ 249.859972] gitaly[904]: time="2026-10-10T11:02:31.081Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN39J2YRXFMH163Q544R8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-10T11:02:41.055" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:31.075" grpc.time_ms=5.8 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1933gitlab # [ 249.874187] gitaly[904]: time="2026-10-10T11:02:31.117Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T11:03:01.116" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:31.117" grpc.time_ms=0.513 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1934gitlab # [ 249.878841] gitaly[904]: time="2026-10-10T11:02:31.122Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4JQN3B0RVPDEXYT1T54TNEM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:41.013" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:31.114" grpc.time_ms=8.031 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1935gitlab # [ 249.898701] gitaly[904]: time="2026-10-10T11:02:31.142Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:01.141" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:31.141" grpc.time_ms=0.429 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1936gitlab # [ 249.911103] gitaly[904]: time="2026-10-10T11:02:31.154Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN3CWBS4FK16DTE2P6KHQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:01.153" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:31.154" grpc.time_ms=0.449 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1937gitlab # [ 249.934343] gitaly[904]: time="2026-10-10T11:02:31.177Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQN39J2YRXFMH163Q544R8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4146 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:31.176" grpc.time_ms=0.722 pid=904 span.kind=server system=grpc1938gitlab # [ 249.937698] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.180Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"219296b25abdcc3cdf1fa014","created_at":"2026-10-10T11:02:31.177Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"09e5490aa8ae07939ceff373","job_size_bytes":3,"pid":2533,"job_status":"deduplicated","message":"PipelineProcessWorker JID-219296b25abdcc3cdf1fa014: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}1939gitlab # [ 249.939119] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.182Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"09e5490aa8ae07939ceff373","created_at":"2026-10-10T11:02:30.873Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-9cb4100418944d4d","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:30.876Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2533,"job_status":"rescheduled","message":"PipelineProcessWorker JID-09e5490aa8ae07939ceff373: rescheduled"}1940gitlab # [ 249.945352] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.188Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"09e5490aa8ae07939ceff373","created_at":"2026-10-10T11:02:30.873Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-9cb4100418944d4d","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:30.876Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-09e5490aa8ae07939ceff373: done: 0.310293 sec","job_status":"done","queue_duration_s":0.001413,"scheduling_latency_s":0.001448,"gitaly_calls":3,"gitaly_duration_s":0.10839,"redis_calls":69,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.015568,"redis_read_bytes":226,"redis_write_bytes":18464,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000383,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001178,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.003883,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12310,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.006795,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000153,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001572,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001221,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":18,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.034,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.110508,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.221085082,"duration_s":0.310293,"completed_at":"2026-10-10T11:02:31.188Z","load_balancing_strategy":"primary","db_duration_s":0.047293,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1941gitlab # [ 249.995782] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.190Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"030721b935c1b408dd226ae0","created_at":"2026-10-10T11:02:30.962Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:30.964Z","job_size_bytes":23,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-030721b935c1b408dd226ae0: start","job_status":"start","queue_duration_s":0.225894,"scheduling_latency_s":0.225927}1942gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses, in 0.25 seconds)1943gitlab # [ 249.996536] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.226Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"030721b935c1b408dd226ae0","created_at":"2026-10-10T11:02:30.962Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:30.964Z","job_size_bytes":23,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-030721b935c1b408dd226ae0: done: 0.036216 sec","job_status":"done","queue_duration_s":0.225894,"scheduling_latency_s":0.225927,"redis_calls":7,"redis_duration_s":0.001155,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000825,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00033,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036216,"completed_at":"2026-10-10T11:02:31.226Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005999,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1944gitlab # [ 250.068330] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fa8a9f48328d3d85c52fff7086159d5273f9ff52 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1861945gitlab # [ 250.069191] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses HTTP/1.1" 200 2 "" "curl/8.21.0" 1311946gitlab # [ 250.069638] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 501947gitlab # [ 250.070157] gitaly[904]: time="2026-10-10T11:02:31.268Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQN3HJNV0RHH19ZZGGEM8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:31.259" grpc.time_ms=9.151 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1948gitlab # [ 250.070718] gitaly[904]: time="2026-10-10T11:02:31.300Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQN39J2YRXFMH163Q544R8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:31.299" grpc.time_ms=0.277 pid=904 span.kind=server system=grpc1949gitlab # [ 250.070902] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.228Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d05628f6d6e3479d3f9d7e24","created_at":"2026-10-10T11:02:31.075Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.081Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d05628f6d6e3479d3f9d7e24: start","job_status":"start","queue_duration_s":0.146049,"scheduling_latency_s":0.146069}1950gitlab # [ 250.071499] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.253Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d05628f6d6e3479d3f9d7e24","created_at":"2026-10-10T11:02:31.075Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.081Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d05628f6d6e3479d3f9d7e24: done: 0.025918 sec","job_status":"done","queue_duration_s":0.146049,"scheduling_latency_s":0.146069,"redis_calls":7,"redis_duration_s":0.001466,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000747,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000719,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025918,"completed_at":"2026-10-10T11:02:31.253Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007894,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1951gitlab # [ 250.145602] gitaly[904]: time="2026-10-10T11:02:31.311Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN39J2YRXFMH163Q544R8 diskcache=5701b428-ab15-4e5c-ae55-fbec04ca0948 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T11:02:41.062" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:02:31.083" pid=904 ref=refs/keep-around/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b remote_ip="2001:db8:1::1" revision=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b span.kind=server system=grpc username=root1952gitlab # [ 250.146470] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.254Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"382c86a90a4e0d7ceefbcdcb","created_at":"2026-10-10T11:02:31.088Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.089Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-382c86a90a4e0d7ceefbcdcb: start","job_status":"start","queue_duration_s":0.165161,"scheduling_latency_s":0.165178}1953gitlab # [ 250.147128] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.277Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"382c86a90a4e0d7ceefbcdcb","created_at":"2026-10-10T11:02:31.088Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.089Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-382c86a90a4e0d7ceefbcdcb: done: 0.022315 sec","job_status":"done","queue_duration_s":0.165161,"scheduling_latency_s":0.165178,"redis_calls":7,"redis_duration_s":0.001134,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000817,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000317,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01368,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022315,"completed_at":"2026-10-10T11:02:31.277Z","load_balancing_strategy":"primary","db_duration_s":0.003867,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1954gitlab # [ 250.189139] gitaly[904]: time="2026-10-10T11:02:31.311Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=81 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5161 command.oublock=8 command.real_time_ms=220 command.spawn_token_fork_ms=0 command.system_time_ms=68 command.user_time_ms=13 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN39J2YRXFMH163Q544R8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-10T11:02:41.062" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-10T11:02:31.083" grpc.time_ms=228.213 pid=904 ref=refs/keep-around/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b remote_ip="2001:db8:1::1" revision=8827c58e7a9316066e7c9f4659b8c5b467dfaf9b span.kind=server system=grpc username=root1955gitlab # [ 250.189729] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.278Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8808737ffb714c7bcf20a8a5","created_at":"2026-10-10T11:02:31.096Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.097Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8808737ffb714c7bcf20a8a5: start","job_status":"start","queue_duration_s":0.180579,"scheduling_latency_s":0.180599}1956gitlab # [ 250.195593] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.305Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8808737ffb714c7bcf20a8a5","created_at":"2026-10-10T11:02:31.096Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.097Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8808737ffb714c7bcf20a8a5: done: 0.027425 sec","job_status":"done","queue_duration_s":0.180579,"scheduling_latency_s":0.180599,"redis_calls":10,"redis_duration_s":0.001554,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000843,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000462,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027425,"completed_at":"2026-10-10T11:02:31.305Z","load_balancing_strategy":"primary","db_duration_s":0.008105,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1957gitlab # [ 250.234696] gitaly[904]: time="2026-10-10T11:02:31.392Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN39J2YRXFMH163Q544R8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:01.390" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:31.391" grpc.time_ms=0.919 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1958gitlab # [ 250.245257] gitaly[904]: time="2026-10-10T11:02:31.438Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN3K6TJ5TZ4FBE769DAF1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:31.428" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1959gitlab # [ 250.254613] gitlab-workhorse[825]: time="2026-10-10T11:02:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671960gitlab # [ 250.256567] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:31 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1681961gitlab # [ 250.258168] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 5251962gitlab # [ 250.261848] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.307Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d92283083ea53c2a4cb7387a","created_at":"2026-10-10T11:02:31.098Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.099Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d92283083ea53c2a4cb7387a: start","job_status":"start","queue_duration_s":0.208108,"scheduling_latency_s":0.208125}1963gitlab # [ 250.308192] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.504Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d92283083ea53c2a4cb7387a","created_at":"2026-10-10T11:02:31.098Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.099Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d92283083ea53c2a4cb7387a: done: 0.197145 sec","job_status":"done","queue_duration_s":0.208108,"scheduling_latency_s":0.208125,"redis_calls":7,"redis_duration_s":0.002914,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002637,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000277,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.177,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.175,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.197145,"completed_at":"2026-10-10T11:02:31.504Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.176481,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1964gitlab # [ 250.348346] gitaly[904]: time="2026-10-10T11:02:31.438Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN3K6TJ5TZ4FBE769DAF1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:31.428" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1965gitlab # [ 250.349105] nixbot[2659]: {"ts": "2026-10-10T11:02:31.503Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-8827c58e7a93-4"}1966gitlab # [ 250.349486] nixbot[2659]: {"ts": "2026-10-10T11:02:31.588Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-8827c58e7a93-4", "argv0": "bwrap"}1967gitlab # [ 250.350062] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.505Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9c0339e5aa5415310005e2f7","created_at":"2026-10-10T11:02:31.099Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.100Z","job_size_bytes":19,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9c0339e5aa5415310005e2f7: start","job_status":"start","queue_duration_s":0.404734,"scheduling_latency_s":0.404755}1968gitlab # [ 250.350624] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.523Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9c0339e5aa5415310005e2f7","created_at":"2026-10-10T11:02:31.099Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.100Z","job_size_bytes":19,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9c0339e5aa5415310005e2f7: done: 0.018067 sec","job_status":"done","queue_duration_s":0.404734,"scheduling_latency_s":0.404755,"redis_calls":8,"redis_duration_s":0.0033989999999999997,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000829,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00257,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018067,"completed_at":"2026-10-10T11:02:31.523Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002628,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1969gitlab # [ 250.389966] gitaly[904]: time="2026-10-10T11:02:31.438Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN3K6TJ5TZ4FBE769DAF1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-10T11:02:31.428" grpc.time_ms=10.145 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc1970gitlab # [ 250.392754] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.524Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3ffd30f5c4d1fcba30db89c0","created_at":"2026-10-10T11:02:31.101Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.102Z","job_size_bytes":66,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3ffd30f5c4d1fcba30db89c0: start","job_status":"start","queue_duration_s":0.422058,"scheduling_latency_s":0.422072}1971gitlab # [ 250.408646] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.541Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3ffd30f5c4d1fcba30db89c0","created_at":"2026-10-10T11:02:31.101Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.102Z","job_size_bytes":66,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3ffd30f5c4d1fcba30db89c0: done: 0.01659 sec","job_status":"done","queue_duration_s":0.422058,"scheduling_latency_s":0.422072,"redis_calls":7,"redis_duration_s":0.00104,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011939,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01659,"completed_at":"2026-10-10T11:02:31.541Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00195,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1972gitlab # [ 250.464202] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.542Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"f75ca7f6087ecb8056d8e7db","created_at":"2026-10-10T11:02:31.104Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.105Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-f75ca7f6087ecb8056d8e7db: start","job_status":"start","queue_duration_s":0.436597,"scheduling_latency_s":0.436619}1973gitlab # [ 250.464854] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.565Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"f75ca7f6087ecb8056d8e7db","created_at":"2026-10-10T11:02:31.104Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.105Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-f75ca7f6087ecb8056d8e7db: done: 0.023299 sec","job_status":"done","queue_duration_s":0.436597,"scheduling_latency_s":0.436619,"redis_calls":7,"redis_duration_s":0.001194,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000866,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000328,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023299,"completed_at":"2026-10-10T11:02:31.565Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00663,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1974gitlab # [ 250.510703] gitaly[904]: time="2026-10-10T11:02:31.656Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:31.656" grpc.time_ms=0.269 pid=904 span.kind=server system=grpc1975gitlab # [ 250.511554] gitaly[904]: time="2026-10-10T11:02:31.674Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3888 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:31.673" grpc.time_ms=0.747 pid=904 span.kind=server system=grpc1976gitlab # [ 250.517057] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.566Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c90d6a1e79c82764da866101","created_at":"2026-10-10T11:02:31.105Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.106Z","job_size_bytes":6,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c90d6a1e79c82764da866101: start","job_status":"start","queue_duration_s":0.459678,"scheduling_latency_s":0.459706}1977gitlab # [ 250.555114] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.576Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"c90d6a1e79c82764da866101","created_at":"2026-10-10T11:02:31.105Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.106Z","job_size_bytes":6,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-c90d6a1e79c82764da866101: done: 0.009552 sec","job_status":"done","queue_duration_s":0.459678,"scheduling_latency_s":0.459706,"redis_calls":7,"redis_duration_s":0.000941,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000673,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009552,"completed_at":"2026-10-10T11:02:31.576Z","load_balancing_strategy":"primary","db_duration_s":0.00027,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1978gitlab # [ 250.601814] gitaly[904]: time="2026-10-10T11:02:31.693Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-10T11:02:31.692" grpc.time_ms=0.335 pid=904 span.kind=server system=grpc1979gitlab # [ 250.602509] gitaly[904]: time="2026-10-10T11:02:31.695Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T11:03:01.633" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:31.634" operations="[repack_refs]" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1980gitlab # [ 250.603063] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.576Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"b45a6fff354f87e1ebc5f1e5","created_at":"2026-10-10T11:02:31.107Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.108Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-b45a6fff354f87e1ebc5f1e5: start","job_status":"start","queue_duration_s":0.468881,"scheduling_latency_s":0.468895}1981gitlab # [ 250.603682] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.592Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"b45a6fff354f87e1ebc5f1e5","created_at":"2026-10-10T11:02:31.107Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.108Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-b45a6fff354f87e1ebc5f1e5: done: 0.015991 sec","job_status":"done","queue_duration_s":0.468881,"scheduling_latency_s":0.468895,"redis_calls":7,"redis_duration_s":0.001009,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000252,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015991,"completed_at":"2026-10-10T11:02:31.592Z","load_balancing_strategy":"primary","db_duration_s":0.002587,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1982gitlab # [ 250.643164] gitaly[904]: time="2026-10-10T11:02:31.697Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T11:03:01.633" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:31.634" pid=904 remote_ip="2001:db8:1::1" repository_info="{false {6 704 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-10-10 11:01:32.458226233 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root1983gitlab # [ 250.645196] gitaly[904]: time="2026-10-10T11:02:31.703Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 diskcache=7e948bbd-6512-4e39-87f5-c052800f3375 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T11:03:01.633" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:31.634" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1984gitlab # [ 250.689331] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.593Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"16ac82811bef0316f3793a5b","created_at":"2026-10-10T11:02:31.183Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.185Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-16ac82811bef0316f3793a5b: start","job_status":"start","queue_duration_s":0.408664,"scheduling_latency_s":0.408682}1985gitlab # [ 250.689899] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.772Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"16ac82811bef0316f3793a5b","created_at":"2026-10-10T11:02:31.183Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-b6b9dc5de8b84e11","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.185Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-16ac82811bef0316f3793a5b: done: 0.178755 sec","job_status":"done","queue_duration_s":0.408664,"scheduling_latency_s":0.408682,"gitaly_calls":3,"gitaly_duration_s":0.074956,"redis_calls":63,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.007312,"redis_read_bytes":144,"redis_write_bytes":16720,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000306,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000601,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":20,"redis_queues_duration_s":0.001639,"redis_queues_read_bytes":24,"redis_queues_write_bytes":11226,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.002611,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00014,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000551,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001158,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077656,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.158226534,"duration_s":0.178755,"completed_at":"2026-10-10T11:02:31.772Z","load_balancing_strategy":"primary","db_duration_s":0.022322,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1986gitlab # [ 250.749967] gitaly[904]: time="2026-10-10T11:02:31.703Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=67 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7563 command.oublock=0 command.real_time_ms=62 command.spawn_token_fork_ms=0 command.system_time_ms=42 command.user_time_ms=25 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T11:03:01.633" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:31.634" grpc.time_ms=69.065 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1987gitlab # [ 250.750645] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.773Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"443a6a8d5c168fe607259305","created_at":"2026-10-10T11:02:31.397Z","trace_propagation_headers":{"sentry-trace":"9d1b176e82ec4de6b4eb3599b7114dfd-ce4b782b30354f8d","baggage":"sentry-trace_id=9d1b176e82ec4de6b4eb3599b7114dfd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQN39J2YRXFMH163Q544R8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.401Z","job_size_bytes":65,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-443a6a8d5c168fe607259305: start","job_status":"start","queue_duration_s":0.372431,"scheduling_latency_s":0.372447}1988gitlab # [ 250.751382] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.791Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"443a6a8d5c168fe607259305","created_at":"2026-10-10T11:02:31.397Z","trace_propagation_headers":{"sentry-trace":"9d1b176e82ec4de6b4eb3599b7114dfd-ce4b782b30354f8d","baggage":"sentry-trace_id=9d1b176e82ec4de6b4eb3599b7114dfd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQN39J2YRXFMH163Q544R8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.401Z","job_size_bytes":65,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-443a6a8d5c168fe607259305: done: 0.017797 sec","job_status":"done","queue_duration_s":0.372431,"scheduling_latency_s":0.372447,"redis_calls":7,"redis_duration_s":0.000902,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000647,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017797,"completed_at":"2026-10-10T11:02:31.791Z","load_balancing_strategy":"primary","db_duration_s":0.002813,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1989gitlab # [ 250.803744] gitaly[904]: time="2026-10-10T11:02:31.712Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T11:03:01.633" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:31.634" pid=904 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root1990gitlab # [ 250.804616] gitaly[904]: time="2026-10-10T11:02:31.712Z" level=info msg="optimized repository" clean_stale_data_time=14.487875ms clean_work_trees_time="77.664µs" commit_graph_time="3.632µs" correlation_id=01M4JQN30HXDCBJK275RWJT7H5 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="4.19µs" pid=904 prune_time="3.632µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="4.191µs" storage_name=default system=housekeeping total_time=14.706059ms1991gitlab # [ 250.805441] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.792Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"80a46b36313886810f306cf2","created_at":"2026-10-10T11:02:31.404Z","trace_propagation_headers":{"sentry-trace":"9d1b176e82ec4de6b4eb3599b7114dfd-ce4b782b30354f8d","baggage":"sentry-trace_id=9d1b176e82ec4de6b4eb3599b7114dfd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQN39J2YRXFMH163Q544R8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.408Z","job_size_bytes":65,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-80a46b36313886810f306cf2: start","job_status":"start","queue_duration_s":0.383519,"scheduling_latency_s":0.383533}1992gitlab # [ 250.843636] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.825Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"80a46b36313886810f306cf2","created_at":"2026-10-10T11:02:31.404Z","trace_propagation_headers":{"sentry-trace":"9d1b176e82ec4de6b4eb3599b7114dfd-ce4b782b30354f8d","baggage":"sentry-trace_id=9d1b176e82ec4de6b4eb3599b7114dfd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQN39J2YRXFMH163Q544R8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.408Z","job_size_bytes":65,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-80a46b36313886810f306cf2: done: 0.033212 sec","job_status":"done","queue_duration_s":0.383519,"scheduling_latency_s":0.383533,"redis_calls":10,"redis_duration_s":0.001379,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000586,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033212,"completed_at":"2026-10-10T11:02:31.825Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007661,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1993gitlab # [ 250.881639] gitaly[904]: time="2026-10-10T11:02:31.712Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-10T11:03:01.633" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:31.634" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1994gitlab # [ 250.882460] gitaly[904]: time="2026-10-10T11:02:31.742Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T11:03:01.741" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:31.742" grpc.time_ms=0.517 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1995gitlab # [ 250.883236] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.826Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"748ce8f255aed75b0ef222da","created_at":"2026-10-10T11:02:31.474Z","trace_propagation_headers":{"sentry-trace":"9d1b176e82ec4de6b4eb3599b7114dfd-ce4b782b30354f8d","baggage":"sentry-trace_id=9d1b176e82ec4de6b4eb3599b7114dfd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQN39J2YRXFMH163Q544R8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.477Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-748ce8f255aed75b0ef222da: start","job_status":"start","queue_duration_s":0.349326,"scheduling_latency_s":0.349349}1996gitlab # [ 250.883788] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.903Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"748ce8f255aed75b0ef222da","created_at":"2026-10-10T11:02:31.474Z","trace_propagation_headers":{"sentry-trace":"9d1b176e82ec4de6b4eb3599b7114dfd-ce4b782b30354f8d","baggage":"sentry-trace_id=9d1b176e82ec4de6b4eb3599b7114dfd,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4JQN39J2YRXFMH163Q544R8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.477Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-748ce8f255aed75b0ef222da: done: 0.077139 sec","job_status":"done","queue_duration_s":0.349326,"scheduling_latency_s":0.349349,"gitaly_calls":2,"gitaly_duration_s":0.00447,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.004104,"redis_read_bytes":80,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000606,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000605,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000176,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001044,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000147,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000259,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000661,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05408,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.056185353,"duration_s":0.077139,"completed_at":"2026-10-10T11:02:31.903Z","load_balancing_strategy":"primary","db_duration_s":0.017475,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1997gitlab # [ 250.924734] gitaly[904]: time="2026-10-10T11:02:31.764Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN30HXDCBJK275RWJT7H5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:01.764" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:31.764" grpc.time_ms=0.38 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1998gitlab # [ 250.968962] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.905Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"96bef8ae86824c9ffb42470a","created_at":"2026-10-10T11:02:31.629Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.630Z","job_size_bytes":23,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-96bef8ae86824c9ffb42470a: start","job_status":"start","queue_duration_s":0.274284,"scheduling_latency_s":0.274305}1999gitlab # [ 250.969650] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.930Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"96bef8ae86824c9ffb42470a","created_at":"2026-10-10T11:02:31.629Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.630Z","job_size_bytes":23,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-96bef8ae86824c9ffb42470a: done: 0.025755 sec","job_status":"done","queue_duration_s":0.274284,"scheduling_latency_s":0.274305,"redis_calls":7,"redis_duration_s":0.000915,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000674,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025755,"completed_at":"2026-10-10T11:02:31.930Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004424,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2000gitlab # [ 251.011568] gitaly[904]: time="2026-10-10T11:02:31.874Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQN39J2YRXFMH163Q544R8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-10T11:03:01.873" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:31.873" grpc.time_ms=0.631 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2001gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses2002gitlab # [ 251.012420] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.931Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"cabcda24702b6f043470cbfe","created_at":"2026-10-10T11:02:31.708Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.710Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-cabcda24702b6f043470cbfe: start","job_status":"start","queue_duration_s":0.220886,"scheduling_latency_s":0.220902}2003gitlab # [ 251.013095] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.946Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"cabcda24702b6f043470cbfe","created_at":"2026-10-10T11:02:31.708Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.710Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-cabcda24702b6f043470cbfe: done: 0.015267 sec","job_status":"done","queue_duration_s":0.220886,"scheduling_latency_s":0.220902,"redis_calls":7,"redis_duration_s":0.000938,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00069,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015267,"completed_at":"2026-10-10T11:02:31.946Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002387,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2004gitlab # [ 251.055970] gitaly[904]: time="2026-10-10T11:02:31.895Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN39J2YRXFMH163Q544R8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:01.894" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:31.894" grpc.time_ms=0.713 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2005gitlab # [ 251.059594] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.947Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f2be814a050239bece501fc1","created_at":"2026-10-10T11:02:31.716Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.718Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f2be814a050239bece501fc1: start","job_status":"start","queue_duration_s":0.228899,"scheduling_latency_s":0.228913}2006gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses, in 0.10 seconds)2007gitlab # [ 251.073404] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.963Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f2be814a050239bece501fc1","created_at":"2026-10-10T11:02:31.716Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.718Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f2be814a050239bece501fc1: done: 0.015773 sec","job_status":"done","queue_duration_s":0.228899,"scheduling_latency_s":0.228913,"redis_calls":7,"redis_duration_s":0.000875,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00064,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000235,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015773,"completed_at":"2026-10-10T11:02:31.963Z","load_balancing_strategy":"primary","db_duration_s":0.002621,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2008gitlab # [ 251.139746] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.964Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4af97a8ad18f679f28807b30","created_at":"2026-10-10T11:02:31.724Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.726Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4af97a8ad18f679f28807b30: start","job_status":"start","queue_duration_s":0.238224,"scheduling_latency_s":0.238238}2009gitlab # [ 251.140572] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.981Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4af97a8ad18f679f28807b30","created_at":"2026-10-10T11:02:31.724Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.726Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4af97a8ad18f679f28807b30: done: 0.017224 sec","job_status":"done","queue_duration_s":0.238224,"scheduling_latency_s":0.238238,"redis_calls":10,"redis_duration_s":0.001246,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000641,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000368,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000237,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017224,"completed_at":"2026-10-10T11:02:31.981Z","load_balancing_strategy":"primary","db_duration_s":0.002524,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2010gitlab # [ 251.182969] gitaly[904]: time="2026-10-10T11:02:32.368Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN4M8Y5TETYW4MXZ68BG9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:02.367" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:32.367" grpc.time_ms=0.667 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2011gitlab # [ 251.184084] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 572012gitlab # [ 251.184650] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.982Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"942066680d640bad045ed2f9","created_at":"2026-10-10T11:02:31.726Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.727Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-942066680d640bad045ed2f9: start","job_status":"start","queue_duration_s":0.255074,"scheduling_latency_s":0.255088}2013gitlab # [ 251.226679] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.996Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"942066680d640bad045ed2f9","created_at":"2026-10-10T11:02:31.726Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.727Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-942066680d640bad045ed2f9: done: 0.014431 sec","job_status":"done","queue_duration_s":0.255074,"scheduling_latency_s":0.255088,"redis_calls":7,"redis_duration_s":0.000899,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000668,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000231,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014431,"completed_at":"2026-10-10T11:02:31.996Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002165,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2014gitlab # [ 251.228120] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:31.997Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"db4ccc8df26d8b1385b715a1","created_at":"2026-10-10T11:02:31.727Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.729Z","job_size_bytes":19,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-db4ccc8df26d8b1385b715a1: start","job_status":"start","queue_duration_s":0.268791,"scheduling_latency_s":0.268805}2015gitlab # [ 251.274717] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:32.011Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"db4ccc8df26d8b1385b715a1","created_at":"2026-10-10T11:02:31.727Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.729Z","job_size_bytes":19,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-db4ccc8df26d8b1385b715a1: done: 0.01371 sec","job_status":"done","queue_duration_s":0.268791,"scheduling_latency_s":0.268805,"redis_calls":8,"redis_duration_s":0.001026,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000765,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01371,"completed_at":"2026-10-10T11:02:32.011Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002051,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2016gitlab # [ 251.279593] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:32.012Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3be3d56a66cc78e0d707cefb","created_at":"2026-10-10T11:02:31.729Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.730Z","job_size_bytes":66,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3be3d56a66cc78e0d707cefb: start","job_status":"start","queue_duration_s":0.281896,"scheduling_latency_s":0.281909}2017gitlab # [ 251.294828] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:32.025Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"3be3d56a66cc78e0d707cefb","created_at":"2026-10-10T11:02:31.729Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.730Z","job_size_bytes":66,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-3be3d56a66cc78e0d707cefb: done: 0.012643 sec","job_status":"done","queue_duration_s":0.281896,"scheduling_latency_s":0.281909,"redis_calls":7,"redis_duration_s":0.000893,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00024,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012643,"completed_at":"2026-10-10T11:02:32.025Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001498,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2018gitlab # [ 251.364700] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:32.026Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"405b4b7d719ecf608f2e7cdf","created_at":"2026-10-10T11:02:31.730Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.731Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-405b4b7d719ecf608f2e7cdf: start","job_status":"start","queue_duration_s":0.294325,"scheduling_latency_s":0.294339}2019gitlab # [ 251.365542] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:32.041Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"405b4b7d719ecf608f2e7cdf","created_at":"2026-10-10T11:02:31.730Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.731Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-405b4b7d719ecf608f2e7cdf: done: 0.015645 sec","job_status":"done","queue_duration_s":0.294325,"scheduling_latency_s":0.294339,"redis_calls":7,"redis_duration_s":0.000913,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000253,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015645,"completed_at":"2026-10-10T11:02:32.041Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003611,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2020gitlab # [ 251.385552] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:32.042Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"dad373dd28f3b8f18acce0cd","created_at":"2026-10-10T11:02:31.731Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.732Z","job_size_bytes":6,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-dad373dd28f3b8f18acce0cd: start","job_status":"start","queue_duration_s":0.30974,"scheduling_latency_s":0.309753}2021gitlab # [ 251.399565] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:32.050Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"dad373dd28f3b8f18acce0cd","created_at":"2026-10-10T11:02:31.731Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.732Z","job_size_bytes":6,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-dad373dd28f3b8f18acce0cd: done: 0.008432 sec","job_status":"done","queue_duration_s":0.30974,"scheduling_latency_s":0.309753,"redis_calls":7,"redis_duration_s":0.0009159999999999999,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000676,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00024,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008432,"completed_at":"2026-10-10T11:02:32.050Z","load_balancing_strategy":"primary","db_duration_s":0.000254,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2022gitlab # [ 251.444883] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:32.051Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"12acf140f74c601ed6c4ce8f","created_at":"2026-10-10T11:02:31.733Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.733Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-12acf140f74c601ed6c4ce8f: start","job_status":"start","queue_duration_s":0.317913,"scheduling_latency_s":0.317927}2023gitlab # [ 251.483600] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:32.065Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"12acf140f74c601ed6c4ce8f","created_at":"2026-10-10T11:02:31.733Z","trace_propagation_headers":{"sentry-trace":"656582010bc942b3a6c4108b457d8d2f-c7c60ab90812434c","baggage":"sentry-trace_id=656582010bc942b3a6c4108b457d8d2f,sentry-release=Unknown"},"correlation_id":"01M4JQN30HXDCBJK275RWJT7H5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.733Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-12acf140f74c601ed6c4ce8f: done: 0.013331 sec","job_status":"done","queue_duration_s":0.317913,"scheduling_latency_s":0.317927,"redis_calls":7,"redis_duration_s":0.000888,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000657,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000231,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013331,"completed_at":"2026-10-10T11:02:32.065Z","load_balancing_strategy":"primary","db_duration_s":0.001966,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2024gitlab # [ 251.485683] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:32.066Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"9eeca5bde514cf4bc57a72bd","created_at":"2026-10-10T11:02:31.863Z","trace_propagation_headers":{"sentry-trace":"9d1b176e82ec4de6b4eb3599b7114dfd-efcdc7cfa5b14f70","baggage":"sentry-trace_id=9d1b176e82ec4de6b4eb3599b7114dfd,sentry-release=Unknown"},"correlation_id":"01M4JQN39J2YRXFMH163Q544R8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.864Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9eeca5bde514cf4bc57a72bd: start","job_status":"start","queue_duration_s":0.201026,"scheduling_latency_s":0.201039}2025gitlab # [ 251.524692] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:32.085Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"9eeca5bde514cf4bc57a72bd","created_at":"2026-10-10T11:02:31.863Z","trace_propagation_headers":{"sentry-trace":"9d1b176e82ec4de6b4eb3599b7114dfd-efcdc7cfa5b14f70","baggage":"sentry-trace_id=9d1b176e82ec4de6b4eb3599b7114dfd,sentry-release=Unknown"},"correlation_id":"01M4JQN39J2YRXFMH163Q544R8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:31.864Z","job_size_bytes":3,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9eeca5bde514cf4bc57a72bd: done: 0.019324 sec","job_status":"done","queue_duration_s":0.201026,"scheduling_latency_s":0.201039,"redis_calls":7,"redis_duration_s":0.000924,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000657,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000267,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014348,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019324,"completed_at":"2026-10-10T11:02:32.085Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002505,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2026gitlab # [ 251.630603] nix-daemon[2824]: accepted connection from pid 3675, user nixbot2027gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses2028gitlab # [ 252.301511] gitaly[904]: time="2026-10-10T11:02:33.544Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN5PWF9Z5WQZXAFHPPBTJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:03.042" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:33.542" grpc.time_ms=1.037 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2029gitlab # [ 252.372445] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:33 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1782030gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses, in 0.23 seconds)2031gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses2032gitlab # [ 253.485451] gitaly[904]: time="2026-10-10T11:02:34.728Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN6X3BJF4KCJ3H3096HZW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:04.726" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:34.726" grpc.time_ms=0.95 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2033gitlab # [ 253.546552] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1292034gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses, in 0.17 seconds)2035gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses2036gitlab # [ 254.651786] gitaly[904]: time="2026-10-10T11:02:35.894Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN81TBJV9SZ1QN7TTN2NR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:05.392" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:35.893" grpc.time_ms=1.012 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2037gitlab # [ 254.686420] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 932038gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses, in 0.14 seconds)2039gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses2040gitlab # [ 255.676715] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:36.919Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"fdc3945ab53caeacba48f5f6","created_at":"2026-10-10T11:02:36.900Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e6574bc475670b4e380713a10801df7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:36.916Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-fdc3945ab53caeacba48f5f6: start","job_status":"start","queue_duration_s":0.002827,"scheduling_latency_s":0.002869}2041gitlab # [ 255.716377] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:36.959Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2042gitlab # [ 255.734471] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:36.977Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"fdc3945ab53caeacba48f5f6","created_at":"2026-10-10T11:02:36.900Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3e6574bc475670b4e380713a10801df7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:36.916Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-fdc3945ab53caeacba48f5f6: done: 0.058067 sec","job_status":"done","queue_duration_s":0.002827,"scheduling_latency_s":0.002869,"redis_calls":9,"redis_duration_s":0.008939,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001017,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003503,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004419,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.058067,"completed_at":"2026-10-10T11:02:36.977Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001506,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2043gitlab # [ 255.778229] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:36.980Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"24192722dce890f115e634a1","created_at":"2026-10-10T11:02:36.931Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef665cf3d443543bf95c42d95aedac53","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:36.945Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-24192722dce890f115e634a1: start","job_status":"start","queue_duration_s":0.034633,"scheduling_latency_s":0.034673}2044gitlab # [ 255.828080] gitaly[904]: time="2026-10-10T11:02:37.070Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN95BNKG5KGXZT31MWWX2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:07.069" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:37.069" grpc.time_ms=0.626 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2045gitlab # [ 255.829340] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:37.070Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"24192722dce890f115e634a1","created_at":"2026-10-10T11:02:36.931Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef665cf3d443543bf95c42d95aedac53","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:36.945Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-24192722dce890f115e634a1: done: 0.089959 sec","job_status":"done","queue_duration_s":0.034633,"scheduling_latency_s":0.034673,"redis_calls":9,"redis_duration_s":0.007638000000000001,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003568,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002354,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001716,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.089959,"completed_at":"2026-10-10T11:02:37.070Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008943,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2046gitlab # [ 255.842799] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1142047gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses, in 0.19 seconds)2048gitlab # [ 255.883942] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:37.127Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ede9ce8a4358427270967bde","created_at":"2026-10-10T11:02:36.966Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd136fab3e0ec5ee110e0d5ccdafc272","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:37.021Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ede9ce8a4358427270967bde: start","job_status":"start","queue_duration_s":0.105359,"scheduling_latency_s":0.105413}2049gitlab # [ 255.936556] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:37.179Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ede9ce8a4358427270967bde","created_at":"2026-10-10T11:02:36.966Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd136fab3e0ec5ee110e0d5ccdafc272","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:37.021Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ede9ce8a4358427270967bde: done: 0.052808 sec","job_status":"done","queue_duration_s":0.105359,"scheduling_latency_s":0.105413,"redis_calls":11,"redis_duration_s":0.014060000000000001,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.006008,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007118,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052808,"completed_at":"2026-10-10T11:02:37.179Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003266,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2050gitlab # [ 255.978438] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:37.221Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"676c3f3ed2427a1f7948bf40","created_at":"2026-10-10T11:02:37.039Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de195f2a9eecdeaaa6cd6e9caf31ccd6","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:37.125Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-676c3f3ed2427a1f7948bf40: start","job_status":"start","queue_duration_s":0.096628,"scheduling_latency_s":0.096659}2051gitlab # [ 256.059558] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:37.302Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"676c3f3ed2427a1f7948bf40","created_at":"2026-10-10T11:02:37.039Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de195f2a9eecdeaaa6cd6e9caf31ccd6","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:37.125Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-676c3f3ed2427a1f7948bf40: done: 0.080498 sec","job_status":"done","queue_duration_s":0.096628,"scheduling_latency_s":0.096659,"redis_calls":12,"redis_duration_s":0.021294999999999998,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00351,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009411,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.008374,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001260775,"exclusive_lock_hold_duration_s":0.013610947,"duration_s":0.080498,"completed_at":"2026-10-10T11:02:37.302Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008343,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2052gitlab # [ 256.095112] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:37.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"ff9209f016096f0628aa2e0c","created_at":"2026-10-10T11:02:37.151Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"879510e92716c87475247131ec572385","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:37.164Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-ff9209f016096f0628aa2e0c: start","job_status":"start","queue_duration_s":0.153592,"scheduling_latency_s":0.153633}2053gitlab # [ 256.111801] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:37.355Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2054gitlab # [ 256.144046] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:37.386Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"ff9209f016096f0628aa2e0c","created_at":"2026-10-10T11:02:37.151Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"879510e92716c87475247131ec572385","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:37.164Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-ff9209f016096f0628aa2e0c: done: 0.068208 sec","job_status":"done","queue_duration_s":0.153592,"scheduling_latency_s":0.153633,"redis_calls":9,"redis_duration_s":0.027105,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002708,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.02313,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001267,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.068208,"completed_at":"2026-10-10T11:02:37.386Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001626,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2055gitlab # [ 256.188423] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:37.431Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"6eb056fd51ea051af787c7ac","created_at":"2026-10-10T11:02:37.172Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b092de5651791f9585c39e9fb2264a96","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:37.223Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-6eb056fd51ea051af787c7ac: start","job_status":"start","queue_duration_s":0.207659,"scheduling_latency_s":0.207707}2056gitlab # [ 256.350206] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:37.593Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"6eb056fd51ea051af787c7ac","created_at":"2026-10-10T11:02:37.172Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b092de5651791f9585c39e9fb2264a96","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:37.223Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-6eb056fd51ea051af787c7ac: done: 0.161773 sec","job_status":"done","queue_duration_s":0.207659,"scheduling_latency_s":0.207707,"redis_calls":12,"redis_duration_s":0.008213999999999999,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001354,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004037,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002823,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.024033222,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.161773,"completed_at":"2026-10-10T11:02:37.592Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016139,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2057gitlab # [ 256.432910] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:37.675Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d8b913a1b1a8072ef7a084a2","created_at":"2026-10-10T11:02:37.252Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41c83a9708800f0b4056439534d221c1","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:37.267Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d8b913a1b1a8072ef7a084a2: start","job_status":"start","queue_duration_s":0.407009,"scheduling_latency_s":0.407043}2058gitlab # [ 256.485710] nix-daemon[2824]: accepted connection from pid 3713, user nixbot2059gitlab # [ 256.501443] nix-daemon[2824]: accepted connection from pid 3675, user nixbot2060gitlab # [ 256.581124] gitaly[904]: time="2026-10-10T11:02:37.823Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQN9YNBN8Y60DAWVECQZE5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:07.822" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:37.822" grpc.time_ms=0.734 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2061gitlab # [ 256.609118] gitaly[904]: time="2026-10-10T11:02:37.852Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQN9YNBN8Y60DAWVECQZE5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:47.256" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:37.847" grpc.time_ms=5.384 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2062gitlab # [ 256.649432] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1102063gitlab # [ 256.690962] gitaly[904]: time="2026-10-10T11:02:37.934Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQNA29QXH1Q9CZYEKVEZY7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:07.033" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:37.933" grpc.time_ms=0.5 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2064gitlab # [ 256.715872] gitaly[904]: time="2026-10-10T11:02:37.959Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQNA29QXH1Q9CZYEKVEZY7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:47.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:37.954" grpc.time_ms=5.146 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2065gitlab # [ 256.750765] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 962066gitlab # [ 256.803457] gitaly[904]: time="2026-10-10T11:02:38.047Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQNA5PXXC8ZJXH039SAHS0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:08.045" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:38.046" grpc.time_ms=0.551 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2067gitlab # [ 256.848981] gitaly[904]: time="2026-10-10T11:02:38.092Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4JQNA5PXXC8ZJXH039SAHS0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-10T11:02:48.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-10T11:02:38.086" grpc.time_ms=5.812 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2068gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses2069gitlab # [ 256.891563] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:38.134Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d8b913a1b1a8072ef7a084a2","created_at":"2026-10-10T11:02:37.252Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41c83a9708800f0b4056439534d221c1","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:37.267Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d8b913a1b1a8072ef7a084a2: done: 0.459453 sec","job_status":"done","queue_duration_s":0.407009,"scheduling_latency_s":0.407043,"redis_calls":564,"redis_duration_s":0.22478499999999998,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003696,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006116,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.214973,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.243021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.459453,"completed_at":"2026-10-10T11:02:38.134Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003627,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2070gitlab # [ 256.930486] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:38.173Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"2c4be89d05447245f60f2003","created_at":"2026-10-10T11:02:37.279Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16d970b220ef6e4de078aa15c01233d5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:37.291Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-2c4be89d05447245f60f2003: start","job_status":"start","queue_duration_s":0.882405,"scheduling_latency_s":0.882444}2071gitlab # [ 256.942784] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1792072gitlab # [ 256.946470] gitaly[904]: time="2026-10-10T11:02:38.189Z" level=info msg="finished unary call with code OK" catfile.duration_ms=5 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=5 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4JQNAA56HBXQEQEZ44C01ND grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-10T11:03:08.083" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-10T11:02:38.184" grpc.time_ms=5.736 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2073gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses, in 0.08 seconds)2074gitlab # [ 256.965454] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/10/10:11:02:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/8827c58e7a9316066e7c9f4659b8c5b467dfaf9b/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 582075(finished: subtest: eval and build post commit statuses, in 7.18 seconds)2076(finished: run the VM test script, in 309.70 seconds)2077gitlab # [ 256.976984] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:38.220Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"2c4be89d05447245f60f2003","created_at":"2026-10-10T11:02:37.279Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16d970b220ef6e4de078aa15c01233d5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:37.291Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-2c4be89d05447245f60f2003: done: 0.046688 sec","job_status":"done","queue_duration_s":0.882405,"scheduling_latency_s":0.882444,"redis_calls":9,"redis_duration_s":0.001606,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000329,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00098,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000297,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046688,"completed_at":"2026-10-10T11:02:38.220Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000893,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2078gitlab # [ 257.019814] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:38.221Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"3f9d839b6545ad9d49905e25","created_at":"2026-10-10T11:02:37.314Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79ed0c57da298f9205505e5646213b56","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:37.325Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-3f9d839b6545ad9d49905e25: start","job_status":"start","queue_duration_s":0.895944,"scheduling_latency_s":0.895965}2079gitlab # [ 257.033381] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:38.228Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2080gitlab # [ 257.037172] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:38.233Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"3f9d839b6545ad9d49905e25","created_at":"2026-10-10T11:02:37.314Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79ed0c57da298f9205505e5646213b56","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:37.325Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-3f9d839b6545ad9d49905e25: done: 0.011799 sec","job_status":"done","queue_duration_s":0.895944,"scheduling_latency_s":0.895965,"redis_calls":9,"redis_duration_s":0.001203,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000261,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000686,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011799,"completed_at":"2026-10-10T11:02:38.233Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000338,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2081gitlab # [ 257.075300] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:38.234Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"5116f846b6007286987b3735","created_at":"2026-10-10T11:02:37.342Z","trace_propagation_headers":{"sentry-trace":"68bce56f7ec244dc866f53a882027d86-168f34cccd614d47","baggage":"sentry-trace_id=68bce56f7ec244dc866f53a882027d86,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f8e2c71e2d35a68e0d32f558cb307f0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-10T11:02:37.356Z","job_size_bytes":2,"pid":2533,"sidekiq_tid":"m7xt","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-5116f846b6007286987b3735: start","job_status":"start","queue_duration_s":0.877188,"scheduling_latency_s":0.877203}2082test script finished in 309.82s2083cleanup2084kill QemuMachine (pid 46)2085gitlab # [ 257.089477] sidekiq-cluster[2533]: {"severity":"INFO","time":"2026-10-10T11:02:38.281Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNameqemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14)2086gitlab # s2087(finished: cleanup, in 1.58 seconds)2088additionally exposed symbols:2089 gitlab,2090 vlan1,2091 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2092[]2093[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-10-10T11:01:43.223Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2094[]2095[]2096[]2097[]2098[]2099[]2100[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '8827c58e7a9316066e7c9f4659b8c5b467dfaf9b', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-10-10T11:02:30.955213Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}]2101[]2102[{'id': 5, 'sha': '8827c58e7a9316066e7c9f4659b8c5b467dfaf9b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-10T11:02:31.470Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2103[{'id': 5, 'sha': '8827c58e7a9316066e7c9f4659b8c5b467dfaf9b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-10T11:02:31.470Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2104[{'id': 5, 'sha': '8827c58e7a9316066e7c9f4659b8c5b467dfaf9b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-10T11:02:31.470Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2105[{'id': 5, 'sha': '8827c58e7a9316066e7c9f4659b8c5b467dfaf9b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-10T11:02:31.470Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2106[{'id': 5, 'sha': '8827c58e7a9316066e7c9f4659b8c5b467dfaf9b', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-10T11:02:31.470Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2107[{'id': 5, 'sha': '8827c58e7a9316066e7c9f4659b8c5b467dfaf9b', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 6s (1 warning)', 'created_at': '2026-10-10T11:02:31.470Z', 'started_at': None, 'finished_at': '2026-10-10T11:02:37.870Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '8827c58e7a9316066e7c9f4659b8c5b467dfaf9b', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-10-10T11:02:37.976Z', 'started_at': None, 'finished_at': '2026-10-10T11:02:38.130Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]