tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab: QEMU running (pid 46) gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab # Formatting '/build/vm-state-gitlab/tmp.yQp3n0VfiY', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) gitlab # Discarding device blocks: 0/262144 done gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: 2fe0dcdf-4451-4271-8b4e-c9feb678cdab gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. (finished: start all VMs, in 49.29 seconds) gitlab: waiting for unit setup-gitlab.service gitlab: waiting for the VM to finish booting gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3inb2m873lcrq0h653vhj9q3ypc2q5n3-nixos-system-gitlab-test/init regInfo=/nix/store/29pyr1xdsv8vapy14s91i3nq3xs9ghr9-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000000] kvm-clock: using sched offset of 430990626 cycles gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000005] tsc: Detected 2400.012 MHz processor gitlab # [ 0.000809] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000836] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000838] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000875] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002726] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002737] Using GB pages for direct mapping gitlab # [ 0.002895] RAMDISK: [mem 0xbe31c000-0xbffcffff] gitlab # [ 0.002902] ACPI: Early table checksum verification disabled gitlab # [ 0.002905] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002909] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002913] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002919] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002921] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.002923] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002924] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002926] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002927] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.002928] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.002929] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.002929] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.002930] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.002930] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003144] No NUMA configuration found gitlab # [ 0.003145] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003147] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff] gitlab # [ 0.005356] Zone ranges: gitlab # [ 0.005356] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.005358] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.005359] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005360] Device empty gitlab # [ 0.005360] Movable zone start for each node gitlab # [ 0.005361] Early memory node ranges gitlab # [ 0.005361] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.005362] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.005363] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005364] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.005383] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.005641] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.110656] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.111068] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.111082] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.111112] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.111115] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.111116] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.111118] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.111119] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.111119] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.111122] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.111123] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.111128] TSC deadline timer available gitlab # [ 0.111132] CPU topo: Max. logical packages: 1 gitlab # [ 0.111133] CPU topo: Max. logical dies: 1 gitlab # [ 0.111133] CPU topo: Max. dies per package: 1 gitlab # [ 0.111137] CPU topo: Max. threads per core: 1 gitlab # [ 0.111138] CPU topo: Num. cores per package: 4 gitlab # [ 0.111139] CPU topo: Num. threads per package: 4 gitlab # [ 0.111139] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.111161] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.111177] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.111179] kvm-guest: setup PV sched yield gitlab # [ 0.111194] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.111195] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.111196] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.111198] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.111199] Booting paravirtualized kernel on KVM gitlab # [ 0.111203] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.115653] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.118041] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.118089] kvm-guest: PV spinlocks enabled gitlab # [ 0.118091] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.118093] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3inb2m873lcrq0h653vhj9q3ypc2q5n3-nixos-system-gitlab-test/init regInfo=/nix/store/29pyr1xdsv8vapy14s91i3nq3xs9ghr9-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.118187] Unknown kernel command line parameters "regInfo=/nix/store/29pyr1xdsv8vapy14s91i3nq3xs9ghr9-closure-info/registration", will be passed to user space. gitlab # [ 0.118201] random: crng init done gitlab # [ 0.118201] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.126759] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.130955] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.131029] software IO TLB: area num 4. gitlab # [ 0.202982] Fallback order for Node 0: 0 gitlab # [ 0.202989] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.202990] Policy zone: Normal gitlab # [ 0.205657] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.215105] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.228329] allocated 12582912 bytes of page_ext gitlab # [ 0.238159] ftrace: allocating 48717 entries in 192 pages gitlab # [ 0.238161] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.239011] Dynamic Preempt: lazy gitlab # [ 0.239172] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.239173] rcu: RCU event tracing is enabled. gitlab # [ 0.239174] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.239175] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.239176] Rude variant of Tasks RCU enabled. gitlab # [ 0.239176] Tracing variant of Tasks RCU enabled. gitlab # [ 0.239177] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.239178] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.239199] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.239200] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.239201] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.243534] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.243853] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.243861] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.243971] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.247534] Console: colour VGA+ 80x25 gitlab # [ 0.247537] printk: legacy console [tty0] enabled gitlab # [ 0.277761] printk: legacy console [ttyS0] enabled gitlab # [ 0.384977] ACPI: Core revision 20250807 gitlab # [ 0.385863] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.387458] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.388488] x2apic enabled gitlab # [ 0.389226] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.390188] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.391425] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.392934] kvm-guest: setup PV IPIs gitlab # [ 0.394519] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.395547] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 0.397298] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) gitlab # [ 0.399379] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.400418] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.402295] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.403299] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.404296] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.405296] Transient Scheduler Attacks: Vulnerable: No microcode gitlab # [ 0.406295] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.407295] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.408295] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.410294] Spectre V2 : Enabling IBPB for BPF gitlab # [ 0.411083] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.412296] active return thunk: srso_alias_return_thunk gitlab # [ 0.413317] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.414295] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.415295] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.417117] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.418127] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.419114] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.420295] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.421296] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.422295] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.423295] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.424295] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.425295] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.426295] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. gitlab # [ 0.456426] Freeing SMP alternatives memory: 44K gitlab # [ 0.457233] pid_max: default: 32768 minimum: 301 gitlab # [ 0.458003] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.459186] landlock: Up and running. gitlab # [ 0.459772] Yama: becoming mindful. gitlab # [ 0.460506] LSM support for eBPF active gitlab # [ 0.461519] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.463391] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.465974] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.466823] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.467300] ... version: 2 gitlab # [ 0.468034] ... bit width: 48 gitlab # [ 0.468298] ... generic counters: 6 gitlab # [ 0.469054] ... generic bitmap: 000000000000003f gitlab # [ 0.469343] ... fixed-purpose counters: 0 gitlab # [ 0.470084] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.470298] ... value mask: 0000ffffffffffff gitlab # [ 0.471237] ... max period: 00007fffffffffff gitlab # [ 0.472043] ... global_ctrl mask: 000000000000003f gitlab # [ 0.472425] signal: max sigframe size: 3376 gitlab # [ 0.473236] rcu: Hierarchical SRCU implementation. gitlab # [ 0.473961] rcu: Max phase no-delay instances is 400. gitlab # [ 0.474471] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.479799] smp: Bringing up secondary CPUs ... gitlab # [ 0.480673] smpboot: x86: Booting SMP configuration: gitlab # [ 0.481306] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.484401] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.486298] smpboot: Total of 4 processors activated (19200.09 BogoMIPS) gitlab # [ 0.487620] Memory: 6011552K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260108K reserved, 0K cma-reserved) gitlab # [ 0.489298] devtmpfs: initialized gitlab # [ 0.490142] x86/mm: Memory block size: 128MB gitlab # [ 0.494341] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.495380] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.496400] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.497530] PM: RTC time: 10:34:47, date: 2026-09-13 gitlab # [ 0.501152] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.503409] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.504350] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.506368] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.507309] audit: initializing netlink subsys (disabled) gitlab # [ 0.508324] audit: type=2000 audit(1789295687.958:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.508560] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.510298] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.511298] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.512298] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.513298] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.514321] cpuidle: using governor menu gitlab # [ 0.516516] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.517535] PCI: Using configuration type 1 for base access gitlab # [ 0.518299] PCI: Using configuration type 1 for extended access gitlab # [ 0.519436] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.521636] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.522299] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.523298] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.524298] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.529329] ACPI: Added _OSI(Module Device) gitlab # [ 0.531310] ACPI: Added _OSI(Processor Device) gitlab # [ 0.533305] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.540149] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.546898] ACPI: Interpreter enabled gitlab # [ 0.548391] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.550306] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.552384] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.556307] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.558648] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.572143] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.573350] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.576930] acpiphp: Slot [3] registered gitlab # [ 0.578343] acpiphp: Slot [4] registered gitlab # [ 0.580341] acpiphp: Slot [5] registered gitlab # [ 0.581357] acpiphp: Slot [6] registered gitlab # [ 0.582341] acpiphp: Slot [7] registered gitlab # [ 0.584340] acpiphp: Slot [8] registered gitlab # [ 0.585355] acpiphp: Slot [9] registered gitlab # [ 0.586343] acpiphp: Slot [10] registered gitlab # [ 0.588340] acpiphp: Slot [11] registered gitlab # [ 0.589341] acpiphp: Slot [12] registered gitlab # [ 0.590356] acpiphp: Slot [13] registered gitlab # [ 0.592343] acpiphp: Slot [14] registered gitlab # [ 0.593320] acpiphp: Slot [15] registered gitlab # [ 0.594107] acpiphp: Slot [16] registered gitlab # [ 0.595326] acpiphp: Slot [17] registered gitlab # [ 0.596081] acpiphp: Slot [18] registered gitlab # [ 0.596317] acpiphp: Slot [19] registered gitlab # [ 0.597057] acpiphp: Slot [20] registered gitlab # [ 0.598346] acpiphp: Slot [21] registered gitlab # [ 0.599093] acpiphp: Slot [22] registered gitlab # [ 0.599317] acpiphp: Slot [23] registered gitlab # [ 0.600080] acpiphp: Slot [24] registered gitlab # [ 0.601340] acpiphp: Slot [25] registered gitlab # [ 0.602090] acpiphp: Slot [26] registered gitlab # [ 0.602318] acpiphp: Slot [27] registered gitlab # [ 0.603079] acpiphp: Slot [28] registered gitlab # [ 0.603319] acpiphp: Slot [29] registered gitlab # [ 0.604068] acpiphp: Slot [30] registered gitlab # [ 0.605317] acpiphp: Slot [31] registered gitlab # [ 0.606070] PCI host bridge to bus 0000:00 gitlab # [ 0.606303] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.608303] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.609298] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.610335] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.611302] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.613330] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.617162] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.621681] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.625442] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.632385] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.634371] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.637303] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.639304] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.642304] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.644773] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.650373] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.654380] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.656850] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.659314] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.662759] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.675468] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.676313] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.677401] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.681344] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.686365] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.697343] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 0.699320] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 0.702380] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 0.731317] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 0.736396] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.744333] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 0.745309] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 0.746321] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 0.752843] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.761344] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 0.763319] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 0.766357] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 0.772381] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.779319] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 0.781333] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 0.783344] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 0.788856] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.796339] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 0.797235] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 0.798088] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 0.800499] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.805332] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 0.806266] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 0.808102] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 0.813517] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.825328] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 0.828322] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 0.831364] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 0.834321] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 0.840790] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.848356] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 0.851365] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 0.857882] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.867346] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 0.870321] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 0.872372] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 0.878205] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.882330] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 0.883250] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 0.884127] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 0.889098] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 0.890462] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 0.892435] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 0.893556] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 0.896450] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 0.900754] iommu: Default domain type: Translated gitlab # [ 0.902303] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 0.905689] ACPI: bus type USB registered gitlab # [ 0.905929] usbcore: registered new interface driver usbfs gitlab # [ 0.907312] usbcore: registered new interface driver hub gitlab # [ 0.908246] usbcore: registered new device driver usb gitlab # [ 0.910234] NetLabel: Initializing gitlab # [ 0.910771] NetLabel: domain hash size = 128 gitlab # [ 0.911298] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 0.912303] NetLabel: unlabeled traffic allowed by default gitlab # [ 0.913240] PCI: Using ACPI for IRQ routing gitlab # [ 0.914546] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 0.915294] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 0.915294] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 0.918303] vgaarb: loaded gitlab # [ 0.919055] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 0.920298] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 0.926372] clocksource: Switched to clocksource kvm-clock gitlab # [ 0.930763] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.931542] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.933187] pnp: PnP ACPI init gitlab # [ 0.934339] pnp: PnP ACPI: found 6 devices gitlab # [ 0.944241] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 0.945921] clocksource: Switched to clocksource acpi_pm gitlab # [ 0.947275] NET: Registered PF_INET protocol family gitlab # [ 0.952328] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.994094] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.995736] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.997192] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 1.000962] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 1.002312] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 1.003557] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 1.004917] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.006343] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 1.007794] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 1.008769] NET: Registered PF_XDP protocol family gitlab # [ 1.009640] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 1.010672] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 1.011692] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 1.012827] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 1.013976] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 1.015343] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 1.016364] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 1.018805] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 1.021304] PCI: CLS 0 bytes, default 64 gitlab # [ 1.022136] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 1.022386] Trying to unpack rootfs image as initramfs... gitlab # [ 1.023255] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB) gitlab # [ 1.023376] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 1.046397] Initialise system trusted keyrings gitlab # [ 1.047771] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 1.059534] Key type asymmetric registered gitlab # [ 1.060326] Asymmetric key parser 'x509' registered gitlab # [ 1.061355] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 1.063294] io scheduler mq-deadline registered gitlab # [ 1.064128] io scheduler kyber registered gitlab # [ 1.067863] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 1.069196] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 1.071782] Linux agpgart interface v0.103 gitlab # [ 1.072569] ACPI: bus type drm_connector registered gitlab # [ 1.074854] usbcore: registered new interface driver usbserial_generic gitlab # [ 1.075984] usbserial: USB Serial support registered for generic gitlab # [ 1.077022] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 1.077022] Please enable it if your BIOS has the CPPC option. gitlab # [ 1.079390] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 1.080879] drop_monitor: Initializing network drop monitor service gitlab # [ 1.082119] NET: Registered PF_INET6 protocol family gitlab # [ 1.083875] Segment Routing with IPv6 gitlab # [ 1.084556] In-situ OAM (IOAM) with IPv6 gitlab # [ 1.087126] IPI shorthand broadcast: enabled gitlab # [ 1.098370] sched_clock: Marking stable (952031995, 145434233)->(1118155619, -20689391) gitlab # [ 1.103181] registered taskstats version 1 gitlab # [ 1.105910] Loading compiled-in X.509 certificates gitlab # [ 1.128686] Demotion targets for Node 0: null gitlab # [ 1.130621] Key type .fscrypt registered gitlab # [ 1.132201] Key type fscrypt-provisioning registered gitlab # [ 1.135000] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 1.137529] ima: Allocated hash algorithm: sha1 gitlab # [ 1.139570] ima: No architecture policies found gitlab # [ 1.141537] PM: Magic number: 10:85:578 gitlab # [ 1.142940] platform pcspkr: hash matches gitlab # [ 1.146045] RAS: Correctable Errors collector initialized. gitlab # [ 1.154617] clk: Disabling unused clocks gitlab # [ 1.155330] PM: genpd: Disabling unused power domains gitlab # [ 1.286602] Freeing initrd memory: 29392K gitlab # [ 1.293767] Freeing unused decrypted memory: 2028K gitlab # [ 1.300005] Freeing unused kernel image (initmem) memory: 3640K gitlab # [ 1.302722] Write protecting the kernel read-only data: 32768k gitlab # [ 1.307676] Freeing unused kernel image (text/rodata gap) memory: 1220K gitlab # [ 1.311548] Freeing unused kernel image (rodata/data gap) memory: 760K gitlab # [ 1.379041] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.380220] Run /init as init process gitlab # [ 1.393182] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.415558] fuse: init (API version 7.45) gitlab # [ 1.424895] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 1.439163] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 1.446638] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 1.510403] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 1.847228] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.887127] systemd[1]: Detected virtualization kvm. gitlab # [ 1.889308] systemd[1]: Detected architecture x86-64. gitlab # [ 1.891496] systemd[1]: Running in initrd. gitlab # [ 1.893963] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.896856] systemd[1]: Hostname set to . gitlab # [ 2.040423] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 2.088624] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 2.098152] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 2.099399] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 2.100877] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 2.102063] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 2.103243] systemd[1]: Reached target Path Units. gitlab # [ 2.104145] systemd[1]: Reached target Slice Units. gitlab # [ 2.105082] systemd[1]: Reached target Swaps. gitlab # [ 2.105937] systemd[1]: Reached target Timer Units. gitlab # [ 2.106935] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 2.108215] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 2.109404] systemd[1]: Listening on Journal Sockets. gitlab # [ 2.110477] systemd[1]: Listening on udev Control Socket. gitlab # [ 2.111547] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 2.112540] systemd[1]: Reached target Socket Units. gitlab # [ 2.114609] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 2.116891] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 2.118939] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 2.121875] systemd[1]: Starting Journal Service... gitlab # [ 2.125530] systemd[1]: Starting Load Kernel Modules... gitlab # [ 2.128527] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 2.137469] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 2.142732] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 2.147147] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.150036] systemd-journald[90]: Collecting audit messages is disabled. gitlab # [ 2.151692] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.155293] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.162762] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 2.183040] netfs: FS-Cache loaded gitlab # [ 2.192897] 9pnet: Installing 9P2000 support gitlab # [ 2.196422] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.199339] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.201950] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.202780] systemd[1]: Started Journal Service. gitlab # [ 2.206168] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.058329] systemd-modules-load[91]: Using 4 probe threads gitlab # [ 2.063853] systemd-modules-load[91]: Inserted module 'virtio_balloon' gitlab # [ 2.065125] systemd-modules-load[91]: Inserted module 'virtio_gpu' gitlab # [ 2.066266] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.067860] systemd-modules-load[91]: Inserted module 'dm_mod' gitlab # [ 2.069097] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.070275] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.071206] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.074082] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 2.075131] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 2.076176] systemd[1]: Reached target Local File Systems. gitlab # [ 2.077865] systemd[1]: Starting Create System Files and Directories... gitlab # [ 2.081124] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 2.083491] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 2.092401] systemd[1]: Finished Create System Files and Directories. gitlab # [ 2.102386] systemd-udevd[111]: Using default interface naming scheme 'v261'. gitlab # [ 2.113217] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 2.152653] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.153604] systemd[1]: Reached target System Initialization. gitlab # [ 2.154449] systemd[1]: Reached target Basic System. gitlab # [ 2.387053] SCSI subsystem initialized gitlab # [ 2.403051] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 2.404261] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 2.405331] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 2.406201] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 2.407345] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 2.408613] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 2.409677] usb usb1: Product: UHCI Host Controller gitlab # [ 2.410362] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd gitlab # [ 2.411168] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 2.435524] scsi host0: ata_piix gitlab # [ 2.436959] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 2.438732] scsi host1: ata_piix gitlab # [ 2.438908] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 2.440691] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 2.441398] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 2.441768] hub 1-0:1.0: USB hub found gitlab # [ 2.443134] hub 1-0:1.0: 2 ports detected gitlab # [ 2.444853] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 2.447952] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 2.449181] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.450910] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 2.455142] virtio_blk virtio5: [vdb] 11843448 512-byte logical blocks (6.06 GB/5.65 GiB) gitlab # [ 2.336554] (udev-worker)[123]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.341362] (udev-worker)[135]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.343543] (udev-worker)[135]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.346340] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.362647] systemd-vconsole-setup[150]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.364501] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.605864] ata2: found unknown device (class 0) gitlab # [ 2.607153] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 2.609855] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 2.646682] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # [ 2.664507] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 2.520909] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.526417] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.528640] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.530682] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.684756] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 2.573250] systemd-fsck[158]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.581153] systemd[1]: Finis[ 2.726402] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # hed File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.831904] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 2.834966] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 2.837607] usb 1-1: Product: QEMU USB Tablet gitlab # [ 2.839233] usb 1-1: Manufacturer: QEMU gitlab # [ 2.840735] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 2.871901] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 2.885183] usbcore: registered new interface driver usbhid gitlab # [ 2.887363] usbhid: USB HID core driver gitlab # [ 2.897416] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 2.902344] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 2.996940] systemd[1]: Mounting /sysroot... gitlab # [ 3.297340] EXT4-fs (vda): mounted filesystem 2fe0dcdf-4451-4271-8b4e-c9feb678cdab r/w with ordered data mode. Quota mode: none. gitlab # [ 3.158959] systemd[1]: Mounted /sysroot. gitlab # [ 3.160026] systemd[1]: Reached target Initrd Root File System. gitlab # [ 3.163740] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 3.165671] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 3.169144] systemd[1]: Mounting /sysroot/run... gitlab # [ 3.184850] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.189790] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.194204] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.200249] systemd[1]: Mounted /sysroot/run. gitlab # [ 3.203930] systemd-sysroot-fstab-check[180]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.211837] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.214978] systemd[1]: Start[ 3.360387] 9p: Installing v9fs 9p2000 file system support gitlab # ing rw-sysroot-nix-store.service... gitlab # [ 3.219121] systemd[1]: Reload requested from client PID 180 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.220917] systemd[1]: Reloading... gitlab # [ 3.367219] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.281462] systemd[1]: Reloading finished in 62 ms. gitlab # [ 3.305235] systemd-sysroot-fstab-check[180]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.310868] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.313240] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 3.315408] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.317649] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.320705] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.326936] systemd-sysroot-fstab-check[180]: Requesting swap.target/start/replace... gitlab # [ 3.332156] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.334509] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.336846] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.338724] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.361548] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.363097] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.997260] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 4.047187] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 4.049130] systemd[1]: Reached target Initrd File Systems. gitlab # [ 4.051114] systemd[1]: Starting Find NixOS closure... gitlab # [ 4.053243] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 4.076487] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 4.078814] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 4.097562] systemd[1]: Finished Find NixOS closure. gitlab # [ 4.099436] systemd[1]: Reached target Initrd Default Target. gitlab # [ 4.101384] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 4.145145] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 4.147290] systemd[1]: Stopped target Basic System. gitlab # [ 4.149102] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 4.150972] systemd[1]: Stopped target Path Units. gitlab # [ 4.152714] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.156512] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.159921] systemd[1]: Stopped target Slice Units. gitlab # [ 4.165572] systemd[1]: Stopped target Socket Units. gitlab # [ 4.168125] systemd[1]: Stopped target System Initialization. gitlab # [ 4.170159] systemd[1]: Stopped target Swaps. gitlab # [ 4.172879] systemd[1]: Stopped target Timer Units. gitlab # [ 4.174716] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.176941] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.179056] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.181525] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.183372] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 4.186938] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.188981] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.191245] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.192854] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.194784] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.196236] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.198339] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.200387] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.202300] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.203963] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.205495] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.207310] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.209241] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.210633] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.212334] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.213492] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.214241] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.215195] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.216113] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.217105] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.217930] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.218822] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.219809] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.220803] systemd[1]: Closed udev Control Socket. gitlab # [ 4.221603] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.222493] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.223552] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.224423] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.225549] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.226528] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.227498] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.242394] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.243485] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.244241] systemd[1]: Reached target Switch Root. gitlab # [ 4.245863] systemd[1]: Starting NixOS Activation... gitlab # [ 4.344881] initrd-nixos-activation-start[230]: booting system configuration /nix/store/3inb2m873lcrq0h653vhj9q3ypc2q5n3-nixos-system-gitlab-test gitlab # [ 4.392232] initrd-nixos-activation-start[230]: running activation script... gitlab # [ 4.605732] initrd-nixos-activation-start[253]: setting up /etc... gitlab # [ 4.725787] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.727393] systemd[1]: Finished NixOS Activation. gitlab # [ 4.729326] systemd[1]: Starting Switch Root... gitlab # [ 4.756360] systemd[1]: Switching root. gitlab # [ 4.965933] systemd-journald[90]: Received SIGTERM from PID 1 (systemd). gitlab # [ 5.041574] NET: Registered PF_VSOCK protocol family gitlab # [ 5.409529] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.413576] systemd[1]: Detected virtualization kvm. gitlab # [ 5.414273] systemd[1]: Detected architecture x86-64. gitlab # [ 5.415106] systemd[1]: Detected first boot. gitlab # [ 5.416239] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.532156] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.621874] systemd[1]: Applying preset policy. gitlab # [ 5.737483] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 5.847476] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 5.848859] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 5.851158] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 5.853191] systemd[1]: Created slice Slice /system/getty. gitlab # [ 5.854579] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 5.856003] systemd[1]: Created slice User and Session Slice. gitlab # [ 5.857189] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 5.858467] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 5.859550] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 5.860280] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 5.861173] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 5.862084] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 5.862930] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 5.863732] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 5.864609] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 5.865507] systemd[1]: Reached target Path Units. gitlab # [ 5.866239] systemd[1]: Reached target Remote File Systems. gitlab # [ 5.867073] systemd[1]: Reached target Slice Units. gitlab # [ 5.867817] systemd[1]: Reached target Swaps. gitlab # [ 5.869217] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 5.871224] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 5.872694] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 5.874220] systemd[1]: Listening on Factory Reset Management. gitlab # [ 5.875201] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 5.877663] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 5.879047] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 5.880955] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 5.882102] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 5.883298] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.884737] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 5.887351] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 5.888441] systemd[1]: Listening on udev Control Socket. gitlab # [ 5.889320] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 5.892070] systemd[1]: Mounting Huge Pages File System... gitlab # [ 5.894138] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 5.896207] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 5.899523] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 5.909998] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 5.911195] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 5.915370] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 5.918146] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 5.922273] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 5.929008] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 5.935399] systemd[1]: Mounting FUSE Control File System... gitlab # [ 5.936365] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 5.956391] systemd[1]: Starting Journal Service... gitlab # [ 5.958402] systemd[1]: Starting Load Kernel Modules... gitlab # [ 5.961544] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 5.964402] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 5.966836] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.975655] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 5.985855] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 5.987132] systemd[1]: Mounted Huge Pages File System. gitlab # [ 5.990659] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 5.991688] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 5.992632] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 5.993300] EXT4-fs (vda): re-mounted 2fe0dcdf-4451-4271-8b4e-c9feb678cdab. gitlab # [ 5.996969] systemd-journald[324]: Collecting audit messages is enabled. gitlab # [ 5.998364] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 6.000235] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 6.001673] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 6.003783] systemd[1]: Mounted FUSE Control File System. gitlab # [ 6.005237] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 6.006794] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 6.009336] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 6.011646] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.011970] loop: module loaded gitlab # [ 5.867947] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 6.013842] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 5.870525] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 5.872796] systemd[1]: systemd-journald.service: Deactivated[ 6.018174] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # successfully. gitlab # [ 6.019878] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.021550] systemd[1]: Started Journal Service. gitlab # [ 5.878784] systemd-modules-load[325]: Using 3 probe threads gitlab # [ 5.880044] systemd-modules-load[325]: Inserted module 'tls' gitlab # [ 5.881434] systemd-oomd[326]: No swap; memory pressure usage will be degraded gitlab # [ 5.883175] systemd-modules-load[325]: Inserted module 'loop' gitlab # [ 5.884769] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 5.886328] systemd[1]: Finished Load Kernel Modules. gitlab # [ 5.888334] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 5.893659] systemd[1]: Starting Firewall... gitlab # [ 5.895485] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 5.899518] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 5.904546] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 5.907835] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 5.909617] systemd[1]: Reached target First Boot Complete. gitlab # [ 5.914178] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 6.061094] systemd-journald[324]: Received client request to flush runtime journal. gitlab # [ 5.951462] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 5.953247] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 5.955525] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 5.956917] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 5.958053] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 5.966929] systemd-udevd[357]: Using default interface naming scheme 'v261'. gitlab # [ 5.982128] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 5.995280] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.073090] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.079298] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 6.118736] (udev-worker)[427]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 6.121102] (udev-worker)[427]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.133796] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 6.135291] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.136814] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.139065] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.140768] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.142054] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.145563] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 6.152813] (udev-worker)[430]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.334169] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 6.336715] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 6.337743] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 6.339765] rtc_cmos 00:05: registered as rtc0 gitlab # [ 6.340591] rtc_cmos 00:05: setting system clock to 2026-09-13T10:34:53 UTC (1789295693) gitlab # [ 6.342237] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 6.342482] Floppy drive(s): gitlab # [ 6.342839] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 6.343338] fd0 is 2.88M AMI BIOS gitlab # [ 6.355686] systemd-journald[324]: Time jumped backwards, rotating. gitlab # [ 6.357931] FDC 0 is a S82078B gitlab # [ 6.371388] ACPI: button: Power Button [PWRF] gitlab # [ 6.373342] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 6.228884] systemd[1]: Finished Firewall. gitlab # [ 6.374335] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 6.402239] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 6.404075] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 6.410606] mousedev: PS/2 mouse device common for all mice gitlab # [ 6.419774] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 gitlab # [ 6.426523] Console: switching to colour dummy device 80x25 gitlab # [ 6.430065] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 6.430067] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 6.433030] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 6.433912] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 6.467354] Console: switching to colour frame buffer device 160x50 gitlab # [ 6.468710] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 6.487264] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6 gitlab # [ 6.364354] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.549769] erofs (device vdb): failed to read inode meta block (nid: 129042425): -4 gitlab # [ 6.554135] kvm_amd: TSC scaling supported gitlab # [ 6.554780] kvm_amd: Nested Virtualization enabled gitlab # [ 6.555244] kvm_amd: Nested Paging enabled gitlab # [ 6.555858] kvm_amd: LBR virtualization supported gitlab # [ 6.556314] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 6.556962] kvm_amd: Virtual GIF supported gitlab # [ 6.557354] kvm_amd: Virtual NMI enabled gitlab # [ 6.412589] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.418311] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.439352] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.586019] ppdev: user-space parallel port driver gitlab # [ 6.602775] EDAC MC: Ver: 3.0.0 gitlab # [ 6.481552] systemd-vconsole-setup[498]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 6.484028] systemd[1]: Finished Virtual Console Setup. gitlab # [ 6.706787] systemd[1]: Mounting /run/wrappers... gitlab # [ 6.733291] systemd[1]: Mounted /run/wrappers. gitlab # [ 6.735072] systemd[1]: Reached target Local File Systems. gitlab # [ 6.736961] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 6.739697] systemd[1]: Starting register-nix-paths.service... gitlab # [ 6.741986] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 6.744086] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.748358] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 6.750685] systemd[1]: Starting Create System Files and Directories... gitlab # [ 6.797351] systemd[1]: Finished Create System Files and Directories. gitlab # [ 6.800702] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 6.803568] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 6.806801] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 6.808816] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 6.832776] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 6.846028] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 6.847946] systemd[1]: Starting Update is Completed... gitlab # [ 6.868389] systemd[1]: Finished Update is Completed. gitlab # [ 6.900484] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 6.901647] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.123816] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.126298] systemd[1]: Reached target System Initialization. gitlab # [ 7.128518] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.131094] systemd[1]: Started logrotate.timer. gitlab # [ 7.133066] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.135467] systemd[1]: Reached target Timer Units. gitlab # [ 7.137398] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.139699] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.141731] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.143849] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.146847] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.149074] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.151576] systemd[1]: Reached target Socket Units. gitlab # [ 7.195322] dbus-broker-launch[580]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.198911] dbus-broker-launch[580]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.201844] dbus-broker-launch[580]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.207760] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.217858] dbus-broker-launch[580]: Ready gitlab # [ 7.220728] systemd[1]: Reached target Basic System. gitlab # [ 7.223655] systemd[1]: Started backdoor.service. gitlab # [ 7.226438] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.230256] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.242968] systemd[1]: Starting Logrotate configuration check... gitlab # [ 7.248365] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.253779] systemd[1]: Starting Post-Boot Actions... gitlab # [ 7.260705] systemd[1]: Started Reset console on configuration changes. gitlab # [ 7.262813] logrotate[585]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # connecting to host... gitlab # [ 7.266076] logrotate[585]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf gitlab # [ 7.267580] logrotate[585]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 7.269069] logrotate[585]: Reading state from file: /var/lib/logrotate.status gitlab # [ 7.269579] logrotate[585]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.271074] logrotate[585]: Allocating hash table for state file, size 64 entries gitlab # [ 7.272096] logrotate[585]: Handling 4 logs gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 7.272375] logrotate[585]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.272992] logrotate[585]: considering log /var/log/btmp gitlab: connected to guest root shell gitlab # [ 7.273960] logrotate[585]: Creating new state gitlab: (connecting took 7.87 seconds) gitlab # [ 7.274382] logrotate[585]: Now: 2026-09-13 10:34 gitlab: (finished: waiting for the VM to finish booting, in 7.87 seconds) gitlab # [ 7.275407] logrotate[585]: Last rotated at 2026-09-13 10:00 gitlab # [ 7.276388] logrotate[585]: log does not need rotating (log has already been rotated) gitlab # [ 7.277363] logrotate[585]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.278320] logrotate[585]: considering log /var/log/wtmp gitlab # [ 7.279247] logrotate[585]: Creating new state gitlab # [ 7.280846] logrotate[585]: Now: 2026-09-13 10:34 gitlab # [ 7.281448] logrotate[585]: Last rotated at 2026-09-13 10:00 gitlab # [ 7.282316] logrotate[585]: log does not need rotating (log has already been rotated) gitlab # [ 7.283218] logrotate[585]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 7.284114] logrotate[585]: switching euid from 0 to 165 and egid from 0 to 165 (pid 585) gitlab # [ 7.284970] logrotate[585]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.285597] logrotate[585]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.286789] logrotate[585]: Creating new state gitlab # [ 7.287933] logrotate[585]: switching euid from 165 to 0 and egid from 165 to 0 (pid 585) gitlab # [ 7.289136] logrotate[585]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.289426] logrotate[585]: switching euid from 0 to 60 and egid from 0 to 60 (pid 585) gitlab # [ 7.290324] logrotate[585]: considering log /var/log/nginx/*.log gitlab # [ 7.291042] logrotate[585]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.291614] logrotate[585]: Creating new state gitlab # [ 7.292183] logrotate[585]: not running postrotate script, since no logs were rotated gitlab # [ 7.292569] logrotate[585]: switching euid from 60 to 0 and egid from 60 to 0 (pid 585) gitlab # [ 7.316745] systemd[1]: Starting resolvconf update... gitlab # [ 7.321409] systemd[1]: Finished Logrotate configuration check. gitlab # [ 7.325124] systemd[1]: Finished Post-Boot Actions. gitlab # [ 7.326041] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.326894] nsncd[586]: Sep 13 10:34:54.628 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.330510] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 7.342957] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.343679] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.347727] systemd[1]: Starting User Login Management... gitlab # [ 7.375881] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.376606] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.377289] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.377730] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.378476] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 7.383427] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 7.383913] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 7.384604] systemd[1]: nscd.service: Consumed 19ms CPU time over 135ms wall clock time, 7.5M memory peak, 5.5M read from disk. gitlab # [ 7.387080] systemd-logind[633]: New seat seat0. gitlab # [ 7.387803] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.394715] systemd-logind[633]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 7.396079] systemd-logind[633]: Watching system buttons on /dev/input/event5 (QEMU Virtio Keyboard) gitlab # [ 7.397368] systemd-logind[633]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 7.398688] systemd[1]: Started User Login Management. gitlab # [ 7.401411] systemd[1]: Starting linger-users.service... gitlab # [ 7.406616] nsncd[677]: Sep 13 10:34:54.711 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.407544] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.408145] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.408674] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.415307] systemd[1]: Finished resolvconf update. gitlab # [ 7.415921] systemd[1]: Reached target Preparation for Network. gitlab # [ 7.420055] systemd[1]: Starting DHCP Client... gitlab # [ 7.422613] systemd[1]: Starting Address configuration of eth1... gitlab # [ 7.425610] systemd[1]: Starting Extra networking commands.... gitlab # [ 7.426577] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 7.430063] systemd[1]: Finished linger-users.service. gitlab # [ 7.454735] network-addresses-eth1-start[703]: adding address 192.168.1.1/24... done gitlab # [ 7.458203] network-addresses-eth1-start[703]: adding address 2001:db8:1::1/64... done gitlab # [ 7.464753] systemd[1]: Finished Address configuration of eth1. gitlab # [ 7.490212] dhcpcd[713]: dhcpcd-10.3.2 starting gitlab # [ 7.492709] dhcpcd[765]: dev: loaded udev gitlab # [ 7.643135] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 7.644107] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 7.501195] systemd[1]: Finished Extra networking commands.. gitlab # [ 7.501943] systemd[1]: Reached target Network. gitlab # [ 7.504854] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 7.506360] systemd[1]: Starting Nginx Web Server... gitlab # [ 7.511716] systemd[1]: Starting PostgreSQL Server... gitlab # [ 7.515366] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 7.517458] systemd[1]: Starting Permit User Sessions... gitlab # [ 7.523407] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 7.537553] systemd[1]: Finished Permit User Sessions. gitlab # [ 7.545338] systemd[1]: Started Getty on tty1. gitlab # [ 7.546434] systemd[1]: Reached target Login Prompts. gitlab # [ 7.737125] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 7.743577] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 7.744641] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 7.746014] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 7.749403] cfg80211: failed to load regulatory.db gitlab # [ 7.771559] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 7.627676] dhcpcd[765]: eth0: waiting for carrier gitlab # [ 7.628628] dhcpcd[765]: eth0: carrier acquired gitlab # [ 7.633773] dhcpcd[765]: DUID 00:01:00:01:32:39:38:ce:52:54:00:12:34:56 gitlab # [ 7.634738] dhcpcd[765]: eth0: IAID 00:12:34:56 gitlab # [ 7.635231] dhcpcd[765]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 7.652237] nginx-pre-start[794]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok gitlab # [ 7.653050] nginx-pre-start[794]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful gitlab # [ 7.657693] systemd[1]: Started Nginx Web Server. gitlab # [ 7.664158] redis-server[793]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 7.667032] redis-server[793]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 7.668834] postgresql-pre-start[798]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 7.670845] postgresql-pre-start[798]: This user must also own the server process. gitlab # [ 7.671957] postgresql-pre-start[798]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 7.673277] postgresql-pre-start[798]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 7.674748] postgresql-pre-start[798]: The default text search configuration will be set to "english". gitlab # [ 7.676247] postgresql-pre-start[798]: Data page checksums are enabled. gitlab # [ 7.677304] postgresql-pre-start[798]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 7.678734] redis-server[793]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=793, just started gitlab # [ 7.680416] postgresql-pre-start[798]: creating subdirectories ... ok gitlab # [ 7.681425] postgresql-pre-start[798]: selecting dynamic shared memory implementation ... posix gitlab # [ 7.682821] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 7.683677] redis-server[793]: Configuration loaded gitlab # [ 7.684896] redis[793]: monotonic clock: POSIX clock_gettime gitlab # [ 7.686137] redis[793]: Running mode=standalone, port=0. gitlab # [ 7.687278] redis[793]: Server initialized gitlab # [ 7.688092] redis[793]: Ready to accept connections unix gitlab # [ 7.689084] redis[793]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 7.700295] postgresql-pre-start[798]: selecting default "max_connections" ... 100 gitlab # [ 7.734776] postgresql-pre-start[798]: selecting default "shared_buffers" ... 128MB gitlab # [ 7.968594] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.117778] postgresql-pre-start[798]: selecting default time zone ... UTC gitlab # [ 8.120198] postgresql-pre-start[798]: creating configuration files ... ok gitlab # [ 8.171106] gitlab-workhorse[829]: time="2026-09-13T10:34:55Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 8.171515] gitlab-workhorse[829]: time="2026-09-13T10:34:55Z" level=info msg="Using redis/go-redis" gitlab # [ 8.171992] gitlab-workhorse[829]: time="2026-09-13T10:34:55Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 8.288370] postgresql-pre-start[798]: running bootstrap script ... ok gitlab # [ 8.708684] postgresql-pre-start[798]: performing post-bootstrap initialization ... ok gitlab # [ 8.917251] postgresql-pre-start[798]: syncing data to disk ... ok gitlab # [ 8.919629] postgresql-pre-start[798]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 8.923075] postgresql-pre-start[798]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 8.928224] postgresql-pre-start[798]: Success. You can now start the database server using: gitlab # [ 8.930917] postgresql-pre-start[798]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 9.002428] postgres[854]: [854] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 9.006994] postgres[854]: [854] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 9.008540] postgres[854]: [854] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 9.015655] postgres[854]: [854] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 9.030066] postgres[863]: [863] LOG: database system was shut down at 2026-09-13 10:34:56 GMT gitlab # [ 9.041882] postgres[854]: [854] LOG: database system is ready to accept connections gitlab # [ 9.051533] systemd[1]: Started PostgreSQL Server. gitlab # [ 9.054365] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 9.239056] postgresql-setup-start[874]: CREATE DATABASE gitlab # [ 9.264345] postgresql-setup-start[884]: CREATE ROLE gitlab # [ 9.276500] postgresql-setup-start[886]: ALTER DATABASE gitlab # [ 9.301307] postgresql-setup-start[891]: CREATE ROLE gitlab # [ 9.303928] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 9.305470] systemd[1]: Reached target PostgreSQL. gitlab # [ 9.307864] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 9.341259] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[879]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 9.342768] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[879]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 9.387288] systemd[1]: Finished gitlab-config.service. gitlab # [ 9.390564] dhcpcd[765]: eth0: soliciting a DHCP lease gitlab # [ 9.538893] NET: Registered PF_PACKET protocol family gitlab # [ 9.394070] gitlab-postgresql-start[899]: CREATE DATABASE gitlab # [ 9.398254] systemd[1]: Started gitaly.service. gitlab # [ 9.401980] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 9.403636] dhcpcd[765]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 9.406397] dhcpcd[765]: eth0: probing address 10.0.2.15/24 gitlab # [ 9.435802] gitlab-postgresql-start[922]: CREATE EXTENSION gitlab # [ 9.489503] dhcpcd[765]: eth0: soliciting an IPv6 router gitlab # [ 9.491761] dhcpcd[765]: eth0: Router Advertisement from fe80::2 gitlab # [ 9.492882] dhcpcd[765]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 9.493961] dhcpcd[765]: eth0: adding route to fec0::/64 gitlab # [ 9.495138] dhcpcd[765]: eth0: adding default route via fe80::2 gitlab # [ 9.519523] gitlab-postgresql-start[932]: CREATE EXTENSION gitlab # [ 9.522762] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 9.959554] gitaly[918]: time="2026-09-13T10:34:57.263Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=918 gitlab # [ 9.961128] gitaly[918]: time="2026-09-13T10:34:57.263Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=918 gitlab # [ 9.962199] gitaly[918]: time="2026-09-13T10:34:57.263Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=918 gitlab # [ 9.963134] gitaly[918]: time="2026-09-13T10:34:57.263Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=918 gitlab # [ 9.963888] gitaly[918]: time="2026-09-13T10:34:57.263Z" level=info msg="Starting Gitaly" pid=918 version= gitlab # [ 10.117899] gitaly[918]: time="2026-09-13T10:34:57.422Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=146 pid=918 gitlab # [ 10.120069] gitaly[918]: time="2026-09-13T10:34:57.423Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=918 gitlab # [ 10.121977] gitaly[918]: time="2026-09-13T10:34:57.424Z" level=info msg="finished initializing command factory" duration_ms=0 pid=918 gitlab # [ 10.123832] gitaly[918]: time="2026-09-13T10:34:57.424Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-918/git-exec-356751007.d/git pid=918 gitlab # [ 10.126106] gitaly[918]: time="2026-09-13T10:34:57.427Z" level=info msg="finished detecting git version" duration_ms=3 pid=918 gitlab # [ 10.127969] gitaly[918]: time="2026-09-13T10:34:57.427Z" level=info msg="using Git version" pid=918 version=2.53.g gitlab # [ 10.129654] gitaly[918]: time="2026-09-13T10:34:57.428Z" level=info msg="clearing disk cache object folder" pid=918 storage=default gitlab # [ 10.131568] gitaly[918]: time="2026-09-13T10:34:57.428Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3651265409 pid=918 storage=default gitlab # [ 10.134428] gitaly[918]: time="2026-09-13T10:34:57.428Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3651265409 pid=918 storage=default gitlab # [ 10.137621] gitaly[918]: time="2026-09-13T10:34:57.428Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=918 gitlab # [ 10.140595] gitaly[918]: time="2026-09-13T10:34:57.428Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=918 gitlab # [ 10.143433] gitaly[918]: time="2026-09-13T10:34:57.428Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3651265409 clear_duration_ms=0 pid=918 storage=default gitlab # [ 10.160387] gitaly[918]: time="2026-09-13T10:34:57.465Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=918 gitlab # [ 10.162890] gitaly[918]: time="2026-09-13T10:34:57.465Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=918 schema=unix gitlab # [ 10.163587] gitaly[918]: time="2026-09-13T10:34:57.465Z" level=info msg="listening at address" address=/run/gitaly/gitaly-918/sock.d/intern pid=918 schema=unix gitlab # [ 10.166080] gitaly[918]: time="2026-09-13T10:34:57.465Z" level=info msg="counting repositories" pid=918 gitlab # [ 10.166671] gitaly[918]: time="2026-09-13T10:34:57.465Z" level=info msg="starting to count repositories" pid=918 storage_path=/var/gitlab/state/repositories gitlab # [ 10.167151] gitaly[918]: time="2026-09-13T10:34:57.465Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=918 storage_path=/var/gitlab/state/repositories gitlab # [ 10.167669] gitaly[918]: time="2026-09-13T10:34:57.465Z" level=info msg="completed counting all repositories" counting_duration=0 pid=918 gitlab # [ 10.168140] gitaly[918]: time="2026-09-13T10:34:57.465Z" level=info msg="finished tempdir cleaner walk" pid=918 storage=default time_ms=0 gitlab # [ 10.168686] gitaly[918]: time="2026-09-13T10:34:57.467Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=918 gitlab # [ 10.169191] gitaly[918]: time="2026-09-13T10:34:57.467Z" level=info msg="Started Gitaly" duration_ms=203 pid=918 gitlab # [ 10.169696] gitaly[918]: time="2026-09-13T10:34:57.467Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=918 gitlab # [ 10.170226] gitaly[918]: time="2026-09-13T10:34:57.469Z" level=info msg="maintenance: daily scheduled" pid=918 scheduled="2026-09-13 12:00:00 +0000 UTC" gitlab # [ 10.718187] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[930]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 10.719051] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[930]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 10.740445] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[930]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.185318] gitaly[918]: time="2026-09-13T10:34:59.489Z" level=info msg="License database preloaded" duration_ms=2225 pid=918 gitlab # [ 14.960885] dhcpcd[765]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 14.962403] dhcpcd[765]: eth0: adding route to 10.0.2.0/24 gitlab # [ 14.963628] dhcpcd[765]: eth0: adding default route via 10.0.2.2 gitlab # [ 15.056629] systemd[1]: Started DHCP Client. gitlab # [ 15.059115] systemd[1]: Reached target Network is Online. gitlab # [ 38.222809] gitlab-workhorse[829]: time="2026-09-13T10:35:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 39.066447] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[930]: W, [2026-09-13T10:35:26.370311 #930] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 41.344963] postgres[1302]: [1302] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 41.347088] postgres[1302]: [1302] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 41.348095] postgres[1302]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 41.349920] postgres[1302]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 41.351897] postgres[1302]: attidentity AS identity, gitlab # [ 41.354223] postgres[1302]: attgenerated as attgenerated gitlab # [ 41.356218] postgres[1302]: FROM pg_attribute a gitlab # [ 41.358190] postgres[1302]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 41.360110] postgres[1302]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 41.361931] postgres[1302]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 41.363892] postgres[1302]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 41.366219] postgres[1302]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 41.368161] postgres[1302]: ORDER BY a.attnum gitlab # [ 41.370126] postgres[1302]: gitlab # [ 44.582782] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1335]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 44.587565] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1335]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 48.232789] gitlab-workhorse[829]: time="2026-09-13T10:35:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 58.233615] gitlab-workhorse[829]: time="2026-09-13T10:35:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 59.359933] postgres[1401]: [1401] ERROR: relation "table_without_models" does not exist gitlab # [ 59.360720] postgres[1401]: [1401] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 68.245080] gitlab-workhorse[829]: time="2026-09-13T10:35:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 70.504539] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 70.506678] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 70.508700] systemd[1]: Starting gitlab.service... gitlab # [ 70.872474] bundle[1568]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 70.874663] bundle[1568]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 70.878387] bundle[1568]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 73.554525] sidekiq-cluster[1567]: {"severity":"INFO","time":"2026-09-13T10:36:00.842Z","message":"Starting cluster with 1 processes"} gitlab # [ 73.649924] bundle[1568]: {"timestamp":"2026-09-13T10:36:00.934Z","pid":1568,"message":"* Enabling systemd notification integration"} gitlab # [ 73.651067] bundle[1568]: {"timestamp":"2026-09-13T10:36:00.953Z","pid":1568,"message":"Puma starting in cluster mode..."} gitlab # [ 73.651858] bundle[1568]: {"timestamp":"2026-09-13T10:36:00.954Z","pid":1568,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 73.652879] bundle[1568]: {"timestamp":"2026-09-13T10:36:00.954Z","pid":1568,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 73.653871] bundle[1568]: {"timestamp":"2026-09-13T10:36:00.954Z","pid":1568,"message":"* Min threads: 0"} gitlab # [ 73.654698] bundle[1568]: {"timestamp":"2026-09-13T10:36:00.954Z","pid":1568,"message":"* Max threads: 4"} gitlab # [ 73.655613] bundle[1568]: {"timestamp":"2026-09-13T10:36:00.954Z","pid":1568,"message":"* Environment: production"} gitlab # [ 73.656608] bundle[1568]: {"timestamp":"2026-09-13T10:36:00.954Z","pid":1568,"message":"* Master PID: 1568"} gitlab # [ 73.657601] bundle[1568]: {"timestamp":"2026-09-13T10:36:00.954Z","pid":1568,"message":"* Workers: 2"} gitlab # [ 73.658110] bundle[1568]: {"timestamp":"2026-09-13T10:36:00.954Z","pid":1568,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 73.658894] bundle[1568]: {"timestamp":"2026-09-13T10:36:00.954Z","pid":1568,"message":"* Preloading application"} gitlab # [ 78.246326] gitlab-workhorse[829]: time="2026-09-13T10:36:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 88.256334] gitlab-workhorse[829]: time="2026-09-13T10:36:15Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 98.266848] gitlab-workhorse[829]: time="2026-09-13T10:36:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 98.457729] sidekiq-cluster[1598]: 2026-09-13T10:36:25.761Z pid=1598 tid=wom INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1598, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>435154944} gitlab # [ 98.625657] sidekiq-cluster[1598]: W, [2026-09-13T10:36:25.930121 #1598] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 98.664205] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:25.968Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 98.964358] bundle[1568]: W, [2026-09-13T10:36:26.268202 #1568] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 100.477421] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.781Z","message":"GitLab reliable fetch activated!"} gitlab # [ 100.481180] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.781Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 100.517298] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.821Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 100.522773] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.823Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 100.528153] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.826Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 100.534549] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.828Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 100.540220] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.830Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 100.545378] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.832Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 100.554334] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.834Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 100.558210] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.836Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 100.562117] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.838Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 100.566370] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.842Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 100.570716] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.844Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 100.573584] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.846Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 100.575571] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.848Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 100.577590] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.849Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 100.579646] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.851Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 100.581778] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.853Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 100.585087] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.855Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 100.587065] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.857Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 100.589234] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.859Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 100.591187] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.861Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 100.593452] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.863Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 100.595802] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.864Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 100.597915] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.866Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 100.600332] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.868Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 100.602557] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.870Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 100.604584] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.872Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 100.606552] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.874Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 100.608602] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.876Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 100.610689] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.878Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 100.612846] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.880Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 100.614869] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.881Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 100.618829] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.883Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 100.620866] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.886Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 100.622996] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.888Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 100.625175] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.890Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 100.627253] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.891Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 100.630985] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.893Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 100.632991] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.895Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 100.635056] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.896Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 100.637506] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.898Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 100.639693] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.900Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 100.641850] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.901Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 100.643969] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.904Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 100.645986] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.908Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 100.648078] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.911Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 100.650358] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.913Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 100.652487] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.915Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 100.654661] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.918Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 100.656679] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.920Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 100.658740] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.922Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 100.660807] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.925Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 100.662890] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.927Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 100.664896] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.937Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 100.667332] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.940Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 100.669421] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.947Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 100.671942] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.949Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 100.672581] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.952Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 100.673393] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.954Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 100.673890] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.956Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 100.674569] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.958Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 100.675019] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.959Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 100.675524] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.961Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 100.675958] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.963Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 100.676623] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.966Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 100.677097] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.967Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 100.677591] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.969Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 100.678049] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.971Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 100.678566] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.972Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 100.679038] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:27.975Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 100.700809] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.005Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 100.705995] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.010Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 100.710672] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.015Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 100.750553] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.055Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 100.756891] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.061Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 100.762308] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.067Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 100.768041] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.072Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 100.774322] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.079Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 100.779798] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.084Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 100.785245] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.089Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 100.791313] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.095Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 100.796678] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.101Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 100.802868] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.107Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 100.809388] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.114Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 100.815498] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.120Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 100.821365] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.126Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 100.827595] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.132Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 100.833934] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.138Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 100.840089] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.144Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 100.845689] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.150Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 100.851788] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.156Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 100.857322] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.161Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 100.863362] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.168Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 100.868739] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.173Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 100.874493] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.179Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 100.880820] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.185Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 100.888057] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.192Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 100.893369] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.198Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 100.899307] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.203Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 100.914523] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.219Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 100.918786] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.223Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 100.924124] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.228Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 100.930446] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.235Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 100.937307] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.241Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 100.942733] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.247Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 100.948447] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.253Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 100.953448] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.258Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 100.958787] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.263Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 100.965582] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.270Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 100.970837] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:28.275Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 104.207786] sidekiq-cluster[1598]: InitializerConnections Query: SET client_min_messages TO 'warning' gitlab # [ 104.208992] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 104.210758] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 104.211756] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 104.213152] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 104.213626] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 104.214321] sidekiq-cluster[1598]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 104.294919] sidekiq-cluster[1598]: InitializerConnections Query: SET standard_conforming_strings = on gitlab # [ 104.295730] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 104.296768] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 104.297528] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 104.298655] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 104.299759] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 104.300858] sidekiq-cluster[1598]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 104.328094] sidekiq-cluster[1598]: InitializerConnections Query: SET intervalstyle = iso_8601 gitlab # [ 104.329301] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 104.330220] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 104.331139] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 104.331742] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 104.332957] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 104.334264] sidekiq-cluster[1598]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 104.369253] sidekiq-cluster[1598]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname gitlab # [ 104.370456] sidekiq-cluster[1598]: FROM pg_type as t gitlab # [ 104.371340] sidekiq-cluster[1598]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz') gitlab # [ 104.371950] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 104.372560] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 104.373451] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 104.374381] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 104.375295] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 104.376366] sidekiq-cluster[1598]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 104.432371] sidekiq-cluster[1598]: InitializerConnections Query: SET SESSION timezone TO 'UTC' gitlab # [ 104.434342] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 104.435358] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 104.436545] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 104.438081] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 104.438800] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 104.440330] sidekiq-cluster[1598]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 104.470102] sidekiq-cluster[1598]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'per_thread_db_connection_pool_metrics' gitlab # [ 104.471600] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 104.473081] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 104.474213] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 104.475343] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 104.476515] sidekiq-cluster[1598]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start' gitlab # [ 104.477979] sidekiq-cluster[1598]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 108.275697] gitlab-workhorse[829]: time="2026-09-13T10:36:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 118.286181] gitlab-workhorse[829]: time="2026-09-13T10:36:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 126.197680] postgres[2045]: [2045] ERROR: relation "table_without_models" does not exist gitlab # [ 126.198896] postgres[2045]: [2045] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 128.288917] gitlab-workhorse[829]: time="2026-09-13T10:36:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 128.811421] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:56.115Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 128.814808] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:56.119Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 128.815467] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:56.119Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 128.816082] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:56.119Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 128.821838] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:56.126Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 129.774458] postgres[2083]: [2083] ERROR: relation "table_without_models" does not exist gitlab # [ 129.775736] postgres[2083]: [2083] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 130.147524] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:57.451Z","message":"Cleaning working queues"} gitlab # [ 130.148692] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:57.452Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"1f0a8a6cc2fec0cab9d3b88f","created_at":"2026-09-13T10:35:57.469Z","trace_propagation_headers":{"sentry-trace":"b0863d2087294249b8472bef955bd5f4-b3ff226487c142fc","baggage":"sentry-trace_id=b0863d2087294249b8472bef955bd5f4,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"5e466307b1f4be975e8ca0fbeca7f033","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T10:35:57.478Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-1f0a8a6cc2fec0cab9d3b88f: start","job_status":"start","queue_duration_s":59.974262,"scheduling_latency_s":59.974309} gitlab # [ 130.327111] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:36:57.631Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"1f0a8a6cc2fec0cab9d3b88f","created_at":"2026-09-13T10:35:57.469Z","trace_propagation_headers":{"sentry-trace":"b0863d2087294249b8472bef955bd5f4-b3ff226487c142fc","baggage":"sentry-trace_id=b0863d2087294249b8472bef955bd5f4,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"5e466307b1f4be975e8ca0fbeca7f033","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T10:35:57.478Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-1f0a8a6cc2fec0cab9d3b88f: done: 0.178671 sec","job_status":"done","queue_duration_s":59.974262,"scheduling_latency_s":59.974309,"redis_calls":31,"redis_duration_s":0.007102,"redis_read_bytes":284,"redis_write_bytes":3897,"redis_feature_flag_calls":20,"redis_feature_flag_duration_s":0.004269,"redis_feature_flag_read_bytes":274,"redis_feature_flag_write_bytes":2650,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001811,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":16,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.087892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.083174817,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.178671,"completed_at":"2026-09-13T10:36:57.631Z","load_balancing_strategy":"primary","db_duration_s":0.043549,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 132.859522] bundle[1568]: {"timestamp":"2026-09-13T10:37:00.163Z","pid":1568,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 132.860839] bundle[1568]: {"timestamp":"2026-09-13T10:37:00.163Z","pid":1568,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 132.861838] bundle[1568]: {"timestamp":"2026-09-13T10:37:00.163Z","pid":1568,"message":"! #\u003cThread:0x00007dd9c5e1fd80 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 132.862761] bundle[1568]: {"timestamp":"2026-09-13T10:37:00.163Z","pid":1568,"message":"! #\u003cThread:0x00007dd9c5ffd0f8 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 132.863604] bundle[1568]: {"timestamp":"2026-09-13T10:37:00.163Z","pid":1568,"message":"Use Ctrl-C to stop"} gitlab # [ 132.873704] bundle[1568]: {"timestamp":"2026-09-13T10:37:00.164Z","pid":1568,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 133.641070] bundle[1568]: {"timestamp":"2026-09-13T10:37:00.945Z","pid":1568,"message":"- Worker 1 (PID: 2179) booted in 0.68s, phase: 0"} gitlab # [ 133.691361] bundle[1568]: {"timestamp":"2026-09-13T10:37:00.995Z","pid":1568,"message":"- Worker 0 (PID: 2175) booted in 0.76s, phase: 0"} gitlab # [ 133.696506] systemd[1]: Started gitlab.service. gitlab # [ 133.697766] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 133.701644] systemd[1]: Starting setup-gitlab.service... gitlab # [ 133.729234] setup-gitlab-start[2212]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 135.849456] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:37:03 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 2091 gitlab # [ 135.856472] setup-gitlab-start[2212]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 135.857640] setup-gitlab-start[2212]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 135.871850] runuser[2230]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 136.269078] setup-gitlab-start[2231]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 136.271151] setup-gitlab-start[2231]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 136.272954] setup-gitlab-start[2231]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 136.860885] setup-gitlab-start[2231]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 136.862115] setup-gitlab-start[2231]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 136.863090] setup-gitlab-start[2231]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 140.927695] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:08.231Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"229921630a5f6f16e87c3fa7","created_at":"2026-09-13T10:37:08.199Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.228Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-229921630a5f6f16e87c3fa7: start","job_status":"start","queue_duration_s":0.002093,"scheduling_latency_s":0.002133} gitlab # [ 141.174631] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:08.478Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"229921630a5f6f16e87c3fa7","created_at":"2026-09-13T10:37:08.199Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.228Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-229921630a5f6f16e87c3fa7: done: 0.247756 sec","job_status":"done","queue_duration_s":0.002093,"scheduling_latency_s":0.002133,"redis_calls":71,"redis_duration_s":0.09256,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.006542,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001481,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.080648,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.003889,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.121244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000693943,"duration_s":0.247756,"completed_at":"2026-09-13T10:37:08.478Z","load_balancing_strategy":"primary","db_duration_s":0.005613,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.248887] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:08.481Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b0534a843ad7d8ffd8b3f4df","created_at":"2026-09-13T10:37:08.236Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba5c3d56ff24ec6de4124b65c54c8f49","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.247Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b0534a843ad7d8ffd8b3f4df: start","job_status":"start","queue_duration_s":0.23399,"scheduling_latency_s":0.234015} gitlab # [ 141.278896] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:08.583Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b0534a843ad7d8ffd8b3f4df","created_at":"2026-09-13T10:37:08.236Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba5c3d56ff24ec6de4124b65c54c8f49","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.247Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b0534a843ad7d8ffd8b3f4df: done: 0.101395 sec","job_status":"done","queue_duration_s":0.23399,"scheduling_latency_s":0.234015,"redis_calls":10,"redis_duration_s":0.009092,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000932,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00353,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00463,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.043,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000657626,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.101395,"completed_at":"2026-09-13T10:37:08.583Z","load_balancing_strategy":"primary","db_duration_s":0.066505,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.300845] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:08.586Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"68d1a8bd70bb256b9540985e","created_at":"2026-09-13T10:37:08.259Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"614df41c4f0badb7149bd614992db301","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.301Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-68d1a8bd70bb256b9540985e: start","job_status":"start","queue_duration_s":0.285455,"scheduling_latency_s":0.28548} gitlab # [ 141.374108] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:08.678Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"68d1a8bd70bb256b9540985e","created_at":"2026-09-13T10:37:08.259Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"614df41c4f0badb7149bd614992db301","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.301Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-68d1a8bd70bb256b9540985e: done: 0.09171 sec","job_status":"done","queue_duration_s":0.285455,"scheduling_latency_s":0.28548,"redis_calls":12,"redis_duration_s":0.011194,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001685,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006561,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002948,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000993143,"exclusive_lock_hold_duration_s":0.007952407,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.09171,"completed_at":"2026-09-13T10:37:08.678Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00597,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.414538] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:08.703Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a909c14d21e951697d6a1908","created_at":"2026-09-13T10:37:08.318Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"162b6fb4f1c4163b0cafab157d96a3cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.331Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a909c14d21e951697d6a1908: start","job_status":"start","queue_duration_s":0.372418,"scheduling_latency_s":0.372501} gitlab # [ 141.453062] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:08.757Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a909c14d21e951697d6a1908","created_at":"2026-09-13T10:37:08.318Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"162b6fb4f1c4163b0cafab157d96a3cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.331Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a909c14d21e951697d6a1908: done: 0.053453 sec","job_status":"done","queue_duration_s":0.372418,"scheduling_latency_s":0.372501,"redis_calls":12,"redis_duration_s":0.003353,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001204,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001447,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000702,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000312331,"duration_s":0.053453,"completed_at":"2026-09-13T10:37:08.757Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001116,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.493503] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:08.797Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"29f3d7380504ae22a204333d","created_at":"2026-09-13T10:37:08.344Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56344fe5fbd45a3be17f2d9678177458","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.354Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-29f3d7380504ae22a204333d: start","job_status":"start","queue_duration_s":0.443167,"scheduling_latency_s":0.4432} gitlab # [ 141.587923] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:08.891Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"29f3d7380504ae22a204333d","created_at":"2026-09-13T10:37:08.344Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56344fe5fbd45a3be17f2d9678177458","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.354Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-29f3d7380504ae22a204333d: done: 0.094075 sec","job_status":"done","queue_duration_s":0.443167,"scheduling_latency_s":0.4432,"redis_calls":12,"redis_duration_s":0.003561,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00043,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001705,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001426,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047214,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000757917,"exclusive_lock_hold_duration_s":0.038774760000000005,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.094075,"completed_at":"2026-09-13T10:37:08.891Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023874,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.658279] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:08.962Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f6200586518a7c526d301070","created_at":"2026-09-13T10:37:08.369Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d4a06f86bb286ace9a37dda42686460","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:37:08.377Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f6200586518a7c526d301070: start","job_status":"start","queue_duration_s":0.584715,"scheduling_latency_s":0.584763} gitlab # [ 141.717187] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.021Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f6200586518a7c526d301070","created_at":"2026-09-13T10:37:08.369Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d4a06f86bb286ace9a37dda42686460","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:37:08.377Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f6200586518a7c526d301070: done: 0.058864 sec","job_status":"done","queue_duration_s":0.584715,"scheduling_latency_s":0.584763,"redis_calls":10,"redis_duration_s":0.002457,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000457,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001503,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000497,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000503137,"duration_s":0.058864,"completed_at":"2026-09-13T10:37:09.021Z","load_balancing_strategy":"primary","db_duration_s":0.006468,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.771720] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.075Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ee13f5b1dc57910afe26aa9e","created_at":"2026-09-13T10:37:08.387Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"48bc97cde72d94bd4c76ea72cb6da22a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:37:08.395Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ee13f5b1dc57910afe26aa9e: start","job_status":"start","queue_duration_s":0.680489,"scheduling_latency_s":0.680532} gitlab # [ 141.801121] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.105Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 141.815095] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.119Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ee13f5b1dc57910afe26aa9e","created_at":"2026-09-13T10:37:08.387Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"48bc97cde72d94bd4c76ea72cb6da22a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:37:08.395Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ee13f5b1dc57910afe26aa9e: done: 0.043665 sec","job_status":"done","queue_duration_s":0.680489,"scheduling_latency_s":0.680532,"redis_calls":10,"redis_duration_s":0.004956,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00095,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003505,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000501,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000510401,"duration_s":0.043665,"completed_at":"2026-09-13T10:37:09.119Z","load_balancing_strategy":"primary","db_duration_s":0.000869,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.855270] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.159Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"11299a7713e0c6b203d12fb2","created_at":"2026-09-13T10:37:08.402Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e0a671f0b3868a16d954a52b49e29cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.410Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-11299a7713e0c6b203d12fb2: start","job_status":"start","queue_duration_s":0.748904,"scheduling_latency_s":0.748944} gitlab # [ 141.874921] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.179Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 141.889695] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.193Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"11299a7713e0c6b203d12fb2","created_at":"2026-09-13T10:37:08.402Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4e0a671f0b3868a16d954a52b49e29cf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.410Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-11299a7713e0c6b203d12fb2: done: 0.034235 sec","job_status":"done","queue_duration_s":0.748904,"scheduling_latency_s":0.748944,"redis_calls":10,"redis_duration_s":0.004901,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000569,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003723,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000609,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000422121,"duration_s":0.034235,"completed_at":"2026-09-13T10:37:09.193Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000787,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 141.931350] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.234Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"de043d0d1f15a40bebfb90cf","created_at":"2026-09-13T10:37:08.419Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0376fcea4a041a9cf4ad87cd6c935dd1","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.428Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-de043d0d1f15a40bebfb90cf: start","job_status":"start","queue_duration_s":0.806593,"scheduling_latency_s":0.806634} gitlab # [ 141.976548] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.280Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"de043d0d1f15a40bebfb90cf","created_at":"2026-09-13T10:37:08.419Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0376fcea4a041a9cf4ad87cd6c935dd1","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.428Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-de043d0d1f15a40bebfb90cf: done: 0.04611 sec","job_status":"done","queue_duration_s":0.806593,"scheduling_latency_s":0.806634,"redis_calls":13,"redis_duration_s":0.003652,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000616,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001915,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001121,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.012914491,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.04611,"completed_at":"2026-09-13T10:37:09.280Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003641,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 142.019543] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.323Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"fcd9a7f3d2f66a9c541c520e","created_at":"2026-09-13T10:37:08.453Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"09998fabd4d25c87f5528ac0aa406bbf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.462Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-fcd9a7f3d2f66a9c541c520e: start","job_status":"start","queue_duration_s":0.861037,"scheduling_latency_s":0.861077} gitlab # [ 142.050240] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.354Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 142.064486] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"fcd9a7f3d2f66a9c541c520e","created_at":"2026-09-13T10:37:08.453Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"09998fabd4d25c87f5528ac0aa406bbf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.462Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-fcd9a7f3d2f66a9c541c520e: done: 0.045277 sec","job_status":"done","queue_duration_s":0.861037,"scheduling_latency_s":0.861077,"redis_calls":10,"redis_duration_s":0.004702,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000457,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003725,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00052,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000418489,"duration_s":0.045277,"completed_at":"2026-09-13T10:37:09.368Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001048,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.106355] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bc52090d87d31e9b5d74773","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bc52090d87d31e9b5d74773: start","job_status":"start","queue_duration_s":0.946167,"scheduling_latency_s":0.946209} gitlab # [ 142.154433] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.458Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bc52090d87d31e9b5d74773","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bc52090d87d31e9b5d74773: done: 0.04811 sec","job_status":"done","queue_duration_s":0.946167,"scheduling_latency_s":0.946209,"redis_calls":16,"redis_duration_s":0.004427,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000464,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001235,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002728,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000843962,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.04811,"completed_at":"2026-09-13T10:37:09.458Z","load_balancing_strategy":"primary","db_duration_s":0.000857,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.193496] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"813a8f9594f6cb651a1408da","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-813a8f9594f6cb651a1408da: start","job_status":"start","queue_duration_s":1.033495,"scheduling_latency_s":1.03353} gitlab # [ 142.231462] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"813a8f9594f6cb651a1408da","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-813a8f9594f6cb651a1408da: done: 0.037702 sec","job_status":"done","queue_duration_s":1.033495,"scheduling_latency_s":1.03353,"redis_calls":13,"redis_duration_s":0.003025,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000995,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016707,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037702,"completed_at":"2026-09-13T10:37:09.535Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.269676] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37b6c93d50fa8d1f0607b039","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37b6c93d50fa8d1f0607b039: start","job_status":"start","queue_duration_s":1.109611,"scheduling_latency_s":1.109647} gitlab # [ 142.308147] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37b6c93d50fa8d1f0607b039","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37b6c93d50fa8d1f0607b039: done: 0.038483 sec","job_status":"done","queue_duration_s":1.109611,"scheduling_latency_s":1.109647,"redis_calls":13,"redis_duration_s":0.004019,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001476,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002543,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038483,"completed_at":"2026-09-13T10:37:09.612Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.346207] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d585b248e9732b7181cc8c1","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d585b248e9732b7181cc8c1: start","job_status":"start","queue_duration_s":1.185764,"scheduling_latency_s":1.185805} gitlab # [ 142.380189] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d585b248e9732b7181cc8c1","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d585b248e9732b7181cc8c1: done: 0.03413 sec","job_status":"done","queue_duration_s":1.185764,"scheduling_latency_s":1.185805,"redis_calls":13,"redis_duration_s":0.002879,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03413,"completed_at":"2026-09-13T10:37:09.684Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.417428] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d7415f430103d1d5a63d06f","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d7415f430103d1d5a63d06f: start","job_status":"start","queue_duration_s":1.257346,"scheduling_latency_s":1.257388} gitlab # [ 142.443118] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d7415f430103d1d5a63d06f","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d7415f430103d1d5a63d06f: done: 0.025309 sec","job_status":"done","queue_duration_s":1.257346,"scheduling_latency_s":1.257388,"redis_calls":13,"redis_duration_s":0.003981999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001229,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002753,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025309,"completed_at":"2026-09-13T10:37:09.746Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.480320] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddfdf11074bb05745235b064","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddfdf11074bb05745235b064: start","job_status":"start","queue_duration_s":1.320083,"scheduling_latency_s":1.320123} gitlab # [ 142.505671] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddfdf11074bb05745235b064","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddfdf11074bb05745235b064: done: 0.025441 sec","job_status":"done","queue_duration_s":1.320083,"scheduling_latency_s":1.320123,"redis_calls":13,"redis_duration_s":0.004405,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025441,"completed_at":"2026-09-13T10:37:09.809Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.507444] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a5df43ee270b44a80c29be3","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a5df43ee270b44a80c29be3: start","job_status":"start","queue_duration_s":1.347203,"scheduling_latency_s":1.347229} gitlab # [ 142.520824] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.825Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a5df43ee270b44a80c29be3","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a5df43ee270b44a80c29be3: done: 0.01388 sec","job_status":"done","queue_duration_s":1.347203,"scheduling_latency_s":1.347229,"redis_calls":13,"redis_duration_s":0.0020369999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000742,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001295,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01388,"completed_at":"2026-09-13T10:37:09.825Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.568773] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bdc7d7d0bab51ee05dd4cf9a","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bdc7d7d0bab51ee05dd4cf9a: start","job_status":"start","queue_duration_s":1.361936,"scheduling_latency_s":1.361948} gitlab # [ 142.569417] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bdc7d7d0bab51ee05dd4cf9a","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bdc7d7d0bab51ee05dd4cf9a: done: 0.009385 sec","job_status":"done","queue_duration_s":1.361936,"scheduling_latency_s":1.361948,"redis_calls":13,"redis_duration_s":0.001558,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001039,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009385,"completed_at":"2026-09-13T10:37:09.835Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.611943] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.836Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d53f2357b5cc9239cad094b3","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d53f2357b5cc9239cad094b3: start","job_status":"start","queue_duration_s":1.372287,"scheduling_latency_s":1.3723} gitlab # [ 142.612738] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d53f2357b5cc9239cad094b3","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d53f2357b5cc9239cad094b3: done: 0.012796 sec","job_status":"done","queue_duration_s":1.372287,"scheduling_latency_s":1.3723,"redis_calls":13,"redis_duration_s":0.002451,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00064,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001811,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012796,"completed_at":"2026-09-13T10:37:09.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.653682] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"192c8f7c45c96f65c29b8b1b","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-192c8f7c45c96f65c29b8b1b: start","job_status":"start","queue_duration_s":1.386496,"scheduling_latency_s":1.386519} gitlab # [ 142.654650] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"192c8f7c45c96f65c29b8b1b","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-192c8f7c45c96f65c29b8b1b: done: 0.015654 sec","job_status":"done","queue_duration_s":1.386496,"scheduling_latency_s":1.386519,"redis_calls":13,"redis_duration_s":0.001879,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000615,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001264,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010927,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015654,"completed_at":"2026-09-13T10:37:09.866Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.695730] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8fbde0b9dc8bd43b7239f38","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8fbde0b9dc8bd43b7239f38: start","job_status":"start","queue_duration_s":1.403544,"scheduling_latency_s":1.403612} gitlab # [ 142.696689] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8fbde0b9dc8bd43b7239f38","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8fbde0b9dc8bd43b7239f38: done: 0.010757 sec","job_status":"done","queue_duration_s":1.403544,"scheduling_latency_s":1.403612,"redis_calls":13,"redis_duration_s":0.0015739999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010757,"completed_at":"2026-09-13T10:37:09.878Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.764755] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f205b2e4568eabbdd4cf3fb","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f205b2e4568eabbdd4cf3fb: start","job_status":"start","queue_duration_s":1.41524,"scheduling_latency_s":1.415252} gitlab # [ 142.765704] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f205b2e4568eabbdd4cf3fb","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f205b2e4568eabbdd4cf3fb: done: 0.009363 sec","job_status":"done","queue_duration_s":1.41524,"scheduling_latency_s":1.415252,"redis_calls":13,"redis_duration_s":0.001349,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000434,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009363,"completed_at":"2026-09-13T10:37:09.888Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.775759] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b887592f142f92502af6050d","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b887592f142f92502af6050d: start","job_status":"start","queue_duration_s":1.4255,"scheduling_latency_s":1.425513} gitlab # [ 142.817050] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b887592f142f92502af6050d","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b887592f142f92502af6050d: done: 0.009023 sec","job_status":"done","queue_duration_s":1.4255,"scheduling_latency_s":1.425513,"redis_calls":13,"redis_duration_s":0.001321,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000426,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009023,"completed_at":"2026-09-13T10:37:09.898Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.820835] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5347ebdd5d35040cc1c0ae61","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5347ebdd5d35040cc1c0ae61: start","job_status":"start","queue_duration_s":1.43529,"scheduling_latency_s":1.435303} gitlab # [ 142.866622] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5347ebdd5d35040cc1c0ae61","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5347ebdd5d35040cc1c0ae61: done: 0.009124 sec","job_status":"done","queue_duration_s":1.43529,"scheduling_latency_s":1.435303,"redis_calls":13,"redis_duration_s":0.00142,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000422,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009124,"completed_at":"2026-09-13T10:37:09.908Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.867635] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.909Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3325f8c2b487e858b042a117","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3325f8c2b487e858b042a117: start","job_status":"start","queue_duration_s":1.445175,"scheduling_latency_s":1.445187} gitlab # [ 142.906260] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3325f8c2b487e858b042a117","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3325f8c2b487e858b042a117: done: 0.009234 sec","job_status":"done","queue_duration_s":1.445175,"scheduling_latency_s":1.445187,"redis_calls":13,"redis_duration_s":0.001531,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000474,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009234,"completed_at":"2026-09-13T10:37:09.918Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.946379] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3e42f25352a01c7597a3c4a","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3e42f25352a01c7597a3c4a: start","job_status":"start","queue_duration_s":1.455364,"scheduling_latency_s":1.455377} gitlab # [ 142.947026] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.928Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3e42f25352a01c7597a3c4a","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3e42f25352a01c7597a3c4a: done: 0.009196 sec","job_status":"done","queue_duration_s":1.455364,"scheduling_latency_s":1.455377,"redis_calls":13,"redis_duration_s":0.001352,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000428,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009196,"completed_at":"2026-09-13T10:37:09.928Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 142.956732] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe989ffaa91cfeaa058746c0","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe989ffaa91cfeaa058746c0: start","job_status":"start","queue_duration_s":1.465345,"scheduling_latency_s":1.465356} gitlab # [ 142.968250] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe989ffaa91cfeaa058746c0","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe989ffaa91cfeaa058746c0: done: 0.009096 sec","job_status":"done","queue_duration_s":1.465345,"scheduling_latency_s":1.465356,"redis_calls":13,"redis_duration_s":0.0013219999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000423,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009096,"completed_at":"2026-09-13T10:37:09.938Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.025737] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6c0421e68a238579450ffb6","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6c0421e68a238579450ffb6: start","job_status":"start","queue_duration_s":1.47521,"scheduling_latency_s":1.475222} gitlab # [ 143.026610] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6c0421e68a238579450ffb6","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6c0421e68a238579450ffb6: done: 0.00918 sec","job_status":"done","queue_duration_s":1.47521,"scheduling_latency_s":1.475222,"redis_calls":13,"redis_duration_s":0.0013180000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000438,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00918,"completed_at":"2026-09-13T10:37:09.948Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.076125] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b42c247bf58eebb74b762da1","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b42c247bf58eebb74b762da1: start","job_status":"start","queue_duration_s":1.485411,"scheduling_latency_s":1.485424} gitlab # [ 143.076846] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b42c247bf58eebb74b762da1","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b42c247bf58eebb74b762da1: done: 0.009072 sec","job_status":"done","queue_duration_s":1.485411,"scheduling_latency_s":1.485424,"redis_calls":13,"redis_duration_s":0.001297,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00042,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000877,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009072,"completed_at":"2026-09-13T10:37:09.958Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.124194] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.959Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6eded357b448adf3b60d56df","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6eded357b448adf3b60d56df: start","job_status":"start","queue_duration_s":1.495434,"scheduling_latency_s":1.495447} gitlab # [ 143.124946] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6eded357b448adf3b60d56df","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6eded357b448adf3b60d56df: done: 0.008969 sec","job_status":"done","queue_duration_s":1.495434,"scheduling_latency_s":1.495447,"redis_calls":13,"redis_duration_s":0.001316,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000425,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006218,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008969,"completed_at":"2026-09-13T10:37:09.968Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.169119] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.969Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afa89028506e4a35327d3303","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afa89028506e4a35327d3303: start","job_status":"start","queue_duration_s":1.505319,"scheduling_latency_s":1.505332} gitlab # [ 143.169908] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afa89028506e4a35327d3303","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afa89028506e4a35327d3303: done: 0.009 sec","job_status":"done","queue_duration_s":1.505319,"scheduling_latency_s":1.505332,"redis_calls":13,"redis_duration_s":0.0013080000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000423,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000885,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009,"completed_at":"2026-09-13T10:37:09.978Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.185578] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"063c8438d43f078cb86391b8","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-063c8438d43f078cb86391b8: start","job_status":"start","queue_duration_s":1.515224,"scheduling_latency_s":1.515237} gitlab # [ 143.196730] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.988Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"063c8438d43f078cb86391b8","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-063c8438d43f078cb86391b8: done: 0.009065 sec","job_status":"done","queue_duration_s":1.515224,"scheduling_latency_s":1.515237,"redis_calls":13,"redis_duration_s":0.001268,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000419,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000849,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009065,"completed_at":"2026-09-13T10:37:09.988Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.238636] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c4ac2985172b2b8f6320625","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c4ac2985172b2b8f6320625: start","job_status":"start","queue_duration_s":1.525195,"scheduling_latency_s":1.525207} gitlab # [ 143.243218] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c4ac2985172b2b8f6320625","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c4ac2985172b2b8f6320625: done: 0.008939 sec","job_status":"done","queue_duration_s":1.525195,"scheduling_latency_s":1.525207,"redis_calls":13,"redis_duration_s":0.0013189999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000442,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000877,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008939,"completed_at":"2026-09-13T10:37:09.998Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.286812] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:09.999Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e21c3932e3d8e95e19105522","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e21c3932e3d8e95e19105522: start","job_status":"start","queue_duration_s":1.534909,"scheduling_latency_s":1.534922} gitlab # [ 143.289083] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.008Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e21c3932e3d8e95e19105522","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e21c3932e3d8e95e19105522: done: 0.009488 sec","job_status":"done","queue_duration_s":1.534909,"scheduling_latency_s":1.534922,"redis_calls":13,"redis_duration_s":0.001531,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009488,"completed_at":"2026-09-13T10:37:10.008Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.336842] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.009Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cb52d6c01cef5a6060efb8c","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cb52d6c01cef5a6060efb8c: start","job_status":"start","queue_duration_s":1.545201,"scheduling_latency_s":1.545213} gitlab # [ 143.337784] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4cb52d6c01cef5a6060efb8c","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4cb52d6c01cef5a6060efb8c: done: 0.009652 sec","job_status":"done","queue_duration_s":1.545201,"scheduling_latency_s":1.545213,"redis_calls":13,"redis_duration_s":0.0015170000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000481,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009652,"completed_at":"2026-09-13T10:37:10.019Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.398597] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c3ef75e3ba4f685e4025eb7","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c3ef75e3ba4f685e4025eb7: start","job_status":"start","queue_duration_s":1.55566,"scheduling_latency_s":1.555672} gitlab # [ 143.399276] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c3ef75e3ba4f685e4025eb7","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c3ef75e3ba4f685e4025eb7: done: 0.00957 sec","job_status":"done","queue_duration_s":1.55566,"scheduling_latency_s":1.555672,"redis_calls":13,"redis_duration_s":0.001817,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000643,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005906,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00957,"completed_at":"2026-09-13T10:37:10.029Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.440762] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb8a0e72350e9f6c1249c2c5","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb8a0e72350e9f6c1249c2c5: start","job_status":"start","queue_duration_s":1.566021,"scheduling_latency_s":1.566034} gitlab # [ 143.441648] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb8a0e72350e9f6c1249c2c5","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb8a0e72350e9f6c1249c2c5: done: 0.009047 sec","job_status":"done","queue_duration_s":1.566021,"scheduling_latency_s":1.566034,"redis_calls":13,"redis_duration_s":0.001356,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000453,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009047,"completed_at":"2026-09-13T10:37:10.039Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.460944] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7c898ab05f7c995f3dbce69","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7c898ab05f7c995f3dbce69: start","job_status":"start","queue_duration_s":1.57585,"scheduling_latency_s":1.575863} gitlab # [ 143.472195] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7c898ab05f7c995f3dbce69","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7c898ab05f7c995f3dbce69: done: 0.008907 sec","job_status":"done","queue_duration_s":1.57585,"scheduling_latency_s":1.575863,"redis_calls":13,"redis_duration_s":0.0013109999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000443,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000868,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008907,"completed_at":"2026-09-13T10:37:10.049Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.522869] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4443be19035d21fc3a62dd7b","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4443be19035d21fc3a62dd7b: start","job_status":"start","queue_duration_s":1.58578,"scheduling_latency_s":1.585793} gitlab # [ 143.523774] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.059Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4443be19035d21fc3a62dd7b","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4443be19035d21fc3a62dd7b: done: 0.009116 sec","job_status":"done","queue_duration_s":1.58578,"scheduling_latency_s":1.585793,"redis_calls":13,"redis_duration_s":0.001423,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000459,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009116,"completed_at":"2026-09-13T10:37:10.059Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.571762] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f2a496107addb62c4560815","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f2a496107addb62c4560815: start","job_status":"start","queue_duration_s":1.595868,"scheduling_latency_s":1.59588} gitlab # [ 143.572485] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f2a496107addb62c4560815","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f2a496107addb62c4560815: done: 0.009468 sec","job_status":"done","queue_duration_s":1.595868,"scheduling_latency_s":1.59588,"redis_calls":13,"redis_duration_s":0.0015049999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000483,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009468,"completed_at":"2026-09-13T10:37:10.069Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.612660] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.070Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f4a1588fc440f5bd6d73e5c","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f4a1588fc440f5bd6d73e5c: start","job_status":"start","queue_duration_s":1.606309,"scheduling_latency_s":1.606322} gitlab # [ 143.613579] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f4a1588fc440f5bd6d73e5c","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f4a1588fc440f5bd6d73e5c: done: 0.009522 sec","job_status":"done","queue_duration_s":1.606309,"scheduling_latency_s":1.606322,"redis_calls":13,"redis_duration_s":0.00162,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009522,"completed_at":"2026-09-13T10:37:10.080Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.655783] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65b72f806bc7bb63b48b2f3e","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65b72f806bc7bb63b48b2f3e: start","job_status":"start","queue_duration_s":1.616837,"scheduling_latency_s":1.61685} gitlab # [ 143.656661] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65b72f806bc7bb63b48b2f3e","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65b72f806bc7bb63b48b2f3e: done: 0.009265 sec","job_status":"done","queue_duration_s":1.616837,"scheduling_latency_s":1.61685,"redis_calls":13,"redis_duration_s":0.0014199999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000454,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006188,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009265,"completed_at":"2026-09-13T10:37:10.090Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.695943] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.091Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cc0648fd35fa4f631610c5e","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cc0648fd35fa4f631610c5e: start","job_status":"start","queue_duration_s":1.627018,"scheduling_latency_s":1.62703} gitlab # [ 143.700960] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cc0648fd35fa4f631610c5e","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cc0648fd35fa4f631610c5e: done: 0.008992 sec","job_status":"done","queue_duration_s":1.627018,"scheduling_latency_s":1.62703,"redis_calls":13,"redis_duration_s":0.0013440000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000432,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008992,"completed_at":"2026-09-13T10:37:10.100Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.747636] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"576fbb4e7320072cd3ee7677","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-576fbb4e7320072cd3ee7677: start","job_status":"start","queue_duration_s":1.636761,"scheduling_latency_s":1.636774} gitlab # [ 143.748259] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"576fbb4e7320072cd3ee7677","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-576fbb4e7320072cd3ee7677: done: 0.009157 sec","job_status":"done","queue_duration_s":1.636761,"scheduling_latency_s":1.636774,"redis_calls":13,"redis_duration_s":0.001327,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000877,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009157,"completed_at":"2026-09-13T10:37:10.110Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.806745] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed33b1ee7467844b3ab30530","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed33b1ee7467844b3ab30530: start","job_status":"start","queue_duration_s":1.646676,"scheduling_latency_s":1.646688} gitlab # [ 143.807439] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed33b1ee7467844b3ab30530","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed33b1ee7467844b3ab30530: done: 0.009181 sec","job_status":"done","queue_duration_s":1.646676,"scheduling_latency_s":1.646688,"redis_calls":13,"redis_duration_s":0.001356,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000436,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009181,"completed_at":"2026-09-13T10:37:10.120Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.851615] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3be780b76e2d9d1428e7e93","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3be780b76e2d9d1428e7e93: start","job_status":"start","queue_duration_s":1.656611,"scheduling_latency_s":1.656623} gitlab # [ 143.852568] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3be780b76e2d9d1428e7e93","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3be780b76e2d9d1428e7e93: done: 0.012681 sec","job_status":"done","queue_duration_s":1.656611,"scheduling_latency_s":1.656623,"redis_calls":13,"redis_duration_s":0.0015999999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012681,"completed_at":"2026-09-13T10:37:10.133Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.906244] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e990dbbb929435d3435eb08","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e990dbbb929435d3435eb08: start","job_status":"start","queue_duration_s":1.670106,"scheduling_latency_s":1.670119} gitlab # [ 143.906973] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e990dbbb929435d3435eb08","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e990dbbb929435d3435eb08: done: 0.009137 sec","job_status":"done","queue_duration_s":1.670106,"scheduling_latency_s":1.670119,"redis_calls":13,"redis_duration_s":0.001344,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000425,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009137,"completed_at":"2026-09-13T10:37:10.143Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.949331] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.144Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1751cd3d6df06c02cf463b8d","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1751cd3d6df06c02cf463b8d: start","job_status":"start","queue_duration_s":1.679991,"scheduling_latency_s":1.680003} gitlab # [ 143.949916] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1751cd3d6df06c02cf463b8d","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1751cd3d6df06c02cf463b8d: done: 0.008931 sec","job_status":"done","queue_duration_s":1.679991,"scheduling_latency_s":1.680003,"redis_calls":13,"redis_duration_s":0.00133,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000458,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000872,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008931,"completed_at":"2026-09-13T10:37:10.153Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 143.963734] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c79458bf54433fcccd6347e","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c79458bf54433fcccd6347e: start","job_status":"start","queue_duration_s":1.6898,"scheduling_latency_s":1.689812} gitlab # [ 144.004622] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.162Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c79458bf54433fcccd6347e","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c79458bf54433fcccd6347e: done: 0.008855 sec","job_status":"done","queue_duration_s":1.6898,"scheduling_latency_s":1.689812,"redis_calls":13,"redis_duration_s":0.001311,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000441,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008855,"completed_at":"2026-09-13T10:37:10.162Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.045115] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fd29e56819478de3c63a036","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fd29e56819478de3c63a036: start","job_status":"start","queue_duration_s":1.699589,"scheduling_latency_s":1.699602} gitlab # [ 144.045900] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fd29e56819478de3c63a036","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fd29e56819478de3c63a036: done: 0.009085 sec","job_status":"done","queue_duration_s":1.699589,"scheduling_latency_s":1.699602,"redis_calls":13,"redis_duration_s":0.001402,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009085,"completed_at":"2026-09-13T10:37:10.172Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.091621] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0682d5450e3ab0d48209988e","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0682d5450e3ab0d48209988e: start","job_status":"start","queue_duration_s":1.709581,"scheduling_latency_s":1.709593} gitlab # [ 144.092252] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0682d5450e3ab0d48209988e","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0682d5450e3ab0d48209988e: done: 0.008901 sec","job_status":"done","queue_duration_s":1.709581,"scheduling_latency_s":1.709593,"redis_calls":13,"redis_duration_s":0.001451,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000888,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008901,"completed_at":"2026-09-13T10:37:10.182Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.099710] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ce65e2284ad830d363a2886","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ce65e2284ad830d363a2886: start","job_status":"start","queue_duration_s":1.719485,"scheduling_latency_s":1.719497} gitlab # [ 144.139280] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ce65e2284ad830d363a2886","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ce65e2284ad830d363a2886: done: 0.008894 sec","job_status":"done","queue_duration_s":1.719485,"scheduling_latency_s":1.719497,"redis_calls":13,"redis_duration_s":0.001459,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000422,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008894,"completed_at":"2026-09-13T10:37:10.192Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.178530] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91a6b25ae7ad670f2730866e","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91a6b25ae7ad670f2730866e: start","job_status":"start","queue_duration_s":1.729339,"scheduling_latency_s":1.729351} gitlab # [ 144.179239] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91a6b25ae7ad670f2730866e","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91a6b25ae7ad670f2730866e: done: 0.008933 sec","job_status":"done","queue_duration_s":1.729339,"scheduling_latency_s":1.729351,"redis_calls":13,"redis_duration_s":0.001339,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00043,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008933,"completed_at":"2026-09-13T10:37:10.202Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.220682] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da78ca04668fcfe95a2cf27b","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da78ca04668fcfe95a2cf27b: start","job_status":"start","queue_duration_s":1.739029,"scheduling_latency_s":1.739042} gitlab # [ 144.221585] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.212Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da78ca04668fcfe95a2cf27b","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da78ca04668fcfe95a2cf27b: done: 0.009077 sec","job_status":"done","queue_duration_s":1.739029,"scheduling_latency_s":1.739042,"redis_calls":13,"redis_duration_s":0.0013449999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009077,"completed_at":"2026-09-13T10:37:10.212Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.236036] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.213Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba0857f36212abdddddfb39e","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba0857f36212abdddddfb39e: start","job_status":"start","queue_duration_s":1.748918,"scheduling_latency_s":1.748932} gitlab # [ 144.247508] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba0857f36212abdddddfb39e","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba0857f36212abdddddfb39e: done: 0.009155 sec","job_status":"done","queue_duration_s":1.748918,"scheduling_latency_s":1.748932,"redis_calls":13,"redis_duration_s":0.001374,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000458,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009155,"completed_at":"2026-09-13T10:37:10.222Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.286759] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3d7a9e997344f0dccd24ed5","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3d7a9e997344f0dccd24ed5: start","job_status":"start","queue_duration_s":1.758843,"scheduling_latency_s":1.758855} gitlab # [ 144.292752] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3d7a9e997344f0dccd24ed5","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3d7a9e997344f0dccd24ed5: done: 0.009196 sec","job_status":"done","queue_duration_s":1.758843,"scheduling_latency_s":1.758855,"redis_calls":13,"redis_duration_s":0.001327,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000454,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009196,"completed_at":"2026-09-13T10:37:10.232Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.332466] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c7485f493bba0680a702dde","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c7485f493bba0680a702dde: start","job_status":"start","queue_duration_s":1.768802,"scheduling_latency_s":1.768815} gitlab # [ 144.374346] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c7485f493bba0680a702dde","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c7485f493bba0680a702dde: done: 0.009004 sec","job_status":"done","queue_duration_s":1.768802,"scheduling_latency_s":1.768815,"redis_calls":13,"redis_duration_s":0.0013109999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000425,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009004,"completed_at":"2026-09-13T10:37:10.242Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.375249] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c3d5d010fbb869d2eef695c","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c3d5d010fbb869d2eef695c: start","job_status":"start","queue_duration_s":1.778532,"scheduling_latency_s":1.778565} gitlab # [ 144.412953] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c3d5d010fbb869d2eef695c","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c3d5d010fbb869d2eef695c: done: 0.009195 sec","job_status":"done","queue_duration_s":1.778532,"scheduling_latency_s":1.778565,"redis_calls":13,"redis_duration_s":0.001461,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000446,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009195,"completed_at":"2026-09-13T10:37:10.252Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.453253] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc6b7eaeca9688b61a78ad01","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc6b7eaeca9688b61a78ad01: start","job_status":"start","queue_duration_s":1.78851,"scheduling_latency_s":1.788523} gitlab # [ 144.453714] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc6b7eaeca9688b61a78ad01","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc6b7eaeca9688b61a78ad01: done: 0.009202 sec","job_status":"done","queue_duration_s":1.78851,"scheduling_latency_s":1.788523,"redis_calls":13,"redis_duration_s":0.00149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009202,"completed_at":"2026-09-13T10:37:10.261Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.462399] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1657fbb09d24d202c4daeaf","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1657fbb09d24d202c4daeaf: start","job_status":"start","queue_duration_s":1.798648,"scheduling_latency_s":1.79866} gitlab # [ 144.506994] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1657fbb09d24d202c4daeaf","created_at":"2026-09-13T10:37:08.274Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-15388e68658a4727","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8e5e1fdb88bea0211744508241d046da","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.464Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1657fbb09d24d202c4daeaf: done: 0.009214 sec","job_status":"done","queue_duration_s":1.798648,"scheduling_latency_s":1.79866,"redis_calls":13,"redis_duration_s":0.001495,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005945,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009214,"completed_at":"2026-09-13T10:37:10.272Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.508056] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.273Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"3d1878a37b61affe6e803f94","created_at":"2026-09-13T10:37:08.471Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2cea1907dd0761bfb00ebddd9c160a4f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.481Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-3d1878a37b61affe6e803f94: start","job_status":"start","queue_duration_s":1.792111,"scheduling_latency_s":1.792124} gitlab # [ 144.520471] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.293Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"3d1878a37b61affe6e803f94","created_at":"2026-09-13T10:37:08.471Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2cea1907dd0761bfb00ebddd9c160a4f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.481Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-3d1878a37b61affe6e803f94: done: 0.020077 sec","job_status":"done","queue_duration_s":1.792111,"scheduling_latency_s":1.792124,"redis_calls":16,"redis_duration_s":0.001948,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.000982,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00025,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011582,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000242489,"duration_s":0.020077,"completed_at":"2026-09-13T10:37:10.293Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003437,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.572561] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.294Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"aaf855fcc80e7c6eb85b2b11","created_at":"2026-09-13T10:37:08.492Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"250b5ab3972b483bcb6ac5551f35b79c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.532Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-aaf855fcc80e7c6eb85b2b11: start","job_status":"start","queue_duration_s":1.761245,"scheduling_latency_s":1.761262} gitlab # [ 144.573150] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.300Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 144.573772] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.305Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"aaf855fcc80e7c6eb85b2b11","created_at":"2026-09-13T10:37:08.492Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"250b5ab3972b483bcb6ac5551f35b79c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.532Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-aaf855fcc80e7c6eb85b2b11: done: 0.011159 sec","job_status":"done","queue_duration_s":1.761245,"scheduling_latency_s":1.761262,"redis_calls":10,"redis_duration_s":0.001068,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000219,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000633,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000216,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006398,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000181029,"duration_s":0.011159,"completed_at":"2026-09-13T10:37:10.305Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000232,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.617639] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.306Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e2f0fbaaddd6f6a591b21049","created_at":"2026-09-13T10:37:08.535Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69620e07fe57278920969ac8820dac2d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.540Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e2f0fbaaddd6f6a591b21049: start","job_status":"start","queue_duration_s":1.765599,"scheduling_latency_s":1.765611} gitlab # [ 144.623023] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.317Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e2f0fbaaddd6f6a591b21049","created_at":"2026-09-13T10:37:08.535Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69620e07fe57278920969ac8820dac2d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.540Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e2f0fbaaddd6f6a591b21049: done: 0.011184 sec","job_status":"done","queue_duration_s":1.765599,"scheduling_latency_s":1.765611,"redis_calls":10,"redis_duration_s":0.0011099999999999999,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000217,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000655,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000238,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000184102,"duration_s":0.011184,"completed_at":"2026-09-13T10:37:10.317Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000213,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.662574] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ee4ab389476a18b7699be466","created_at":"2026-09-13T10:37:08.549Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f6f1705b804d98e83b137f22e536f27","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.556Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ee4ab389476a18b7699be466: start","job_status":"start","queue_duration_s":1.761511,"scheduling_latency_s":1.761524} gitlab # [ 144.672931] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ee4ab389476a18b7699be466","created_at":"2026-09-13T10:37:08.549Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f6f1705b804d98e83b137f22e536f27","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.556Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ee4ab389476a18b7699be466: done: 0.135242 sec","job_status":"done","queue_duration_s":1.761511,"scheduling_latency_s":1.761524,"redis_calls":14,"redis_duration_s":0.0018960000000000001,"redis_read_bytes":13,"redis_write_bytes":1641,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000605,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00059,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.024,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.09662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000412342,"exclusive_lock_hold_duration_s":0.12119585,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.135242,"completed_at":"2026-09-13T10:37:10.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04149,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.732715] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"25b16b55628a94f08aade61f","created_at":"2026-09-13T10:37:08.567Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a6c42a53738accfd130ebdf796ebef6f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.574Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-25b16b55628a94f08aade61f: start","job_status":"start","queue_duration_s":1.879949,"scheduling_latency_s":1.879963} gitlab # [ 144.733464] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.469Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"25b16b55628a94f08aade61f","created_at":"2026-09-13T10:37:08.567Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a6c42a53738accfd130ebdf796ebef6f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.574Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-25b16b55628a94f08aade61f: done: 0.014995 sec","job_status":"done","queue_duration_s":1.879949,"scheduling_latency_s":1.879963,"redis_calls":10,"redis_duration_s":0.0011560000000000001,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000226,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000195276,"duration_s":0.014995,"completed_at":"2026-09-13T10:37:10.469Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001546,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.777779] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.470Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"6a85ca1dfd709240c0af661f","created_at":"2026-09-13T10:37:08.584Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ea995761e6c40d7cdecdc273dc6faee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.591Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-6a85ca1dfd709240c0af661f: start","job_status":"start","queue_duration_s":1.878988,"scheduling_latency_s":1.879} gitlab # [ 144.778671] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.484Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 144.779179] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.491Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"6a85ca1dfd709240c0af661f","created_at":"2026-09-13T10:37:08.584Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ea995761e6c40d7cdecdc273dc6faee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.591Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-6a85ca1dfd709240c0af661f: done: 0.021473 sec","job_status":"done","queue_duration_s":1.878988,"scheduling_latency_s":1.879,"redis_calls":10,"redis_duration_s":0.01064,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001451,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004852,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004337,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000964368,"duration_s":0.021473,"completed_at":"2026-09-13T10:37:10.491Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000777,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.830291] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.495Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"256b9d0cdf941d649b7a126e","created_at":"2026-09-13T10:37:08.598Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20e308ffd3f72ac7daa945ef45d517ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.602Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-256b9d0cdf941d649b7a126e: start","job_status":"start","queue_duration_s":1.892867,"scheduling_latency_s":1.89288} gitlab # [ 144.831078] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.520Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"256b9d0cdf941d649b7a126e","created_at":"2026-09-13T10:37:08.598Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20e308ffd3f72ac7daa945ef45d517ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.602Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-256b9d0cdf941d649b7a126e: done: 0.025718 sec","job_status":"done","queue_duration_s":1.892867,"scheduling_latency_s":1.89288,"redis_calls":12,"redis_duration_s":0.003583,"redis_read_bytes":11,"redis_write_bytes":1612,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000759,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001328,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001496,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000323225,"exclusive_lock_hold_duration_s":0.01028818,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.025718,"completed_at":"2026-09-13T10:37:10.520Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005235,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.871783] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.521Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"c235d666afc0375be8d48bdf","created_at":"2026-09-13T10:37:08.641Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"30a254576b62c496837668ea14022a4d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.650Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-c235d666afc0375be8d48bdf: start","job_status":"start","queue_duration_s":1.871562,"scheduling_latency_s":1.871574} gitlab # [ 144.910958] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.542Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"c235d666afc0375be8d48bdf","created_at":"2026-09-13T10:37:08.641Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"30a254576b62c496837668ea14022a4d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.650Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-c235d666afc0375be8d48bdf: done: 0.020495 sec","job_status":"done","queue_duration_s":1.871562,"scheduling_latency_s":1.871574,"redis_calls":19,"redis_duration_s":0.002058,"redis_read_bytes":21,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000447,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000171,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001078,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000362,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0001936,"duration_s":0.020495,"completed_at":"2026-09-13T10:37:10.542Z","load_balancing_strategy":"primary","db_duration_s":0.00111,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 144.952911] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.548Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"697d1776c56b37ab63d87b91","created_at":"2026-09-13T10:37:08.662Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6eb4e8f7b966c0b608c55a9c320339c3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:37:08.675Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-697d1776c56b37ab63d87b91: start","job_status":"start","queue_duration_s":1.87292,"scheduling_latency_s":1.872934} gitlab # [ 144.953590] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.601Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 144.954129] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"697d1776c56b37ab63d87b91","created_at":"2026-09-13T10:37:08.662Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6eb4e8f7b966c0b608c55a9c320339c3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:37:08.675Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-697d1776c56b37ab63d87b91: done: 0.058267 sec","job_status":"done","queue_duration_s":1.87292,"scheduling_latency_s":1.872934,"redis_calls":10,"redis_duration_s":0.006973,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001334,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005333,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208685,"duration_s":0.058267,"completed_at":"2026-09-13T10:37:10.606Z","load_balancing_strategy":"primary","db_duration_s":0.002284,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.005240] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.607Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"83d72f0d257fb799d79fcfdd","created_at":"2026-09-13T10:37:08.699Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b6cefe7f666e6664c9a938958cd5a4f5","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.704Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-83d72f0d257fb799d79fcfdd: start","job_status":"start","queue_duration_s":1.902425,"scheduling_latency_s":1.902438} gitlab # [ 145.005970] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.834Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"83d72f0d257fb799d79fcfdd","created_at":"2026-09-13T10:37:08.699Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b6cefe7f666e6664c9a938958cd5a4f5","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.704Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-83d72f0d257fb799d79fcfdd: done: 0.227271 sec","job_status":"done","queue_duration_s":1.902425,"scheduling_latency_s":1.902438,"redis_calls":565,"redis_duration_s":0.07638099999999999,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000217,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000721,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.075443,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.173092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000224609,"duration_s":0.227271,"completed_at":"2026-09-13T10:37:10.834Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.044715] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"54dac310f2cf9857325c276d","created_at":"2026-09-13T10:37:08.706Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9bef082144a272f44d6a54e42afb5b5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.710Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-54dac310f2cf9857325c276d: start","job_status":"start","queue_duration_s":2.125271,"scheduling_latency_s":2.125286} gitlab # [ 145.045873] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.852Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"54dac310f2cf9857325c276d","created_at":"2026-09-13T10:37:08.706Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-82c4d52a625845ff","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9bef082144a272f44d6a54e42afb5b5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:37:08.710Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-54dac310f2cf9857325c276d: done: 0.016519 sec","job_status":"done","queue_duration_s":2.125271,"scheduling_latency_s":2.125286,"redis_calls":12,"redis_duration_s":0.001558,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00025,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000714,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000594,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009372,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000313168,"exclusive_lock_hold_duration_s":0.00409689,"duration_s":0.016519,"completed_at":"2026-09-13T10:37:10.851Z","load_balancing_strategy":"primary","db_duration_s":0.00201,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.093875] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.852Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"578d1b4565c9fb8e117476c1","created_at":"2026-09-13T10:37:10.530Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-0e860b1afed94f20","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"30a254576b62c496837668ea14022a4d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:10.537Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-578d1b4565c9fb8e117476c1: start","job_status":"start","queue_duration_s":0.315331,"scheduling_latency_s":0.315345} gitlab # [ 145.094578] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.861Z","message":"No existing merge request to be cleaned up."} gitlab # [ 145.095105] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.869Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"578d1b4565c9fb8e117476c1","created_at":"2026-09-13T10:37:10.530Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-0e860b1afed94f20","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"30a254576b62c496837668ea14022a4d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:10.537Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-578d1b4565c9fb8e117476c1: done: 0.016609 sec","job_status":"done","queue_duration_s":0.315331,"scheduling_latency_s":0.315345,"redis_calls":13,"redis_duration_s":0.001604,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000617,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000734,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000193041,"duration_s":0.016609,"completed_at":"2026-09-13T10:37:10.869Z","load_balancing_strategy":"primary","db_duration_s":0.001904,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.135673] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.870Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"80c57e8fb69856d1ba0dc570","created_at":"2026-09-13T10:37:10.530Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-0e860b1afed94f20","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"30a254576b62c496837668ea14022a4d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:10.537Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-80c57e8fb69856d1ba0dc570: start","job_status":"start","queue_duration_s":0.332731,"scheduling_latency_s":0.332743} gitlab # [ 145.136239] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.876Z","message":"No existing merge request to be cleaned up."} gitlab # [ 145.136773] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.880Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"80c57e8fb69856d1ba0dc570","created_at":"2026-09-13T10:37:10.530Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-0e860b1afed94f20","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"30a254576b62c496837668ea14022a4d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:10.537Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-80c57e8fb69856d1ba0dc570: done: 0.010549 sec","job_status":"done","queue_duration_s":0.332731,"scheduling_latency_s":0.332743,"redis_calls":10,"redis_duration_s":0.0012439999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000743,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010549,"completed_at":"2026-09-13T10:37:10.880Z","load_balancing_strategy":"primary","db_duration_s":0.001064,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.187321] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.881Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d74056fc27e076366a349686","created_at":"2026-09-13T10:37:10.530Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-0e860b1afed94f20","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"30a254576b62c496837668ea14022a4d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:10.537Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d74056fc27e076366a349686: start","job_status":"start","queue_duration_s":0.344109,"scheduling_latency_s":0.344122} gitlab # [ 145.188034] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.887Z","message":"No existing merge request to be cleaned up."} gitlab # [ 145.188674] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.891Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d74056fc27e076366a349686","created_at":"2026-09-13T10:37:10.530Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-0e860b1afed94f20","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"30a254576b62c496837668ea14022a4d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:10.537Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d74056fc27e076366a349686: done: 0.010157 sec","job_status":"done","queue_duration_s":0.344109,"scheduling_latency_s":0.344122,"redis_calls":10,"redis_duration_s":0.001214,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000726,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010157,"completed_at":"2026-09-13T10:37:10.891Z","load_balancing_strategy":"primary","db_duration_s":0.000938,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.232021] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.892Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"08278061f93691449028c0e3","created_at":"2026-09-13T10:37:10.530Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-0e860b1afed94f20","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"30a254576b62c496837668ea14022a4d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:10.537Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-08278061f93691449028c0e3: start","job_status":"start","queue_duration_s":0.355042,"scheduling_latency_s":0.355055} gitlab # [ 145.232710] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.898Z","message":"No existing merge request to be cleaned up."} gitlab # [ 145.233235] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:10.902Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"08278061f93691449028c0e3","created_at":"2026-09-13T10:37:10.530Z","trace_propagation_headers":{"sentry-trace":"9a902a1aac2547b586270684a8b24077-0e860b1afed94f20","baggage":"sentry-trace_id=9a902a1aac2547b586270684a8b24077,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"30a254576b62c496837668ea14022a4d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:37:10.537Z","job_size_bytes":2,"pid":1598,"sidekiq_tid":"mahm","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-08278061f93691449028c0e3: done: 0.010169 sec","job_status":"done","queue_duration_s":0.355042,"scheduling_latency_s":0.355055,"redis_calls":10,"redis_duration_s":0.001232,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000736,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010169,"completed_at":"2026-09-13T10:37:10.902Z","load_balancing_strategy":"primary","db_duration_s":0.000893,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 145.438395] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2033217097 wd_nsec: 2033216754 gitlab # [ 162.372877] setup-gitlab-start[2231]: W, [2026-09-13T10:37:29.676820 #2231] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 186.143694] postgres[2447]: [2447] ERROR: relation "table_without_models" does not exist gitlab # [ 186.145370] postgres[2447]: [2447] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 190.216712] runuser[2230]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 190.218772] setup-gitlab-start[2212]: + systemctl restart gitlab-sidekiq.service gitlab # [ 190.243330] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 190.312369] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:57.616Z","message":"Shutting down"} gitlab # [ 190.313750] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:57.616Z","message":"Scheduler exiting..."} gitlab # [ 190.314842] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:57.616Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 190.315691] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:57.616Z","message":"Scheduler exiting..."} gitlab # [ 190.813305] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:37:58.117Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 194.318818] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:38:01.622Z","message":"Bye!"} gitlab # [ 194.348896] sidekiq-cluster[1598]: {"severity":"INFO","time":"2026-09-13T10:38:01.653Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1598,"worker_id":"sidekiq_0","memwd_rss_bytes":935129088,"retry":0} gitlab # [ 195.590182] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 195.592719] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 195.594677] systemd[1]: gitlab-sidekiq.service: Consumed 57.278s CPU time over 2min 5.083s wall clock time, 1013.5M memory peak, 13.7M read from disk. gitlab # [ 195.599442] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 195.601923] setup-gitlab-start[2212]: + mkdir -p /var/lib/secrets gitlab # [ 195.604227] setup-gitlab-start[2212]: + echo nixbot-test-token-of-20-chars gitlab # [ 195.606466] setup-gitlab-start[2212]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 195.608853] setup-gitlab-start[2212]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars gitlab # [ 195.911224] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:03 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 272 gitlab # [ 196.477124] gitaly[918]: time="2026-09-13T10:38:03.780Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:38:13.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:03.780" pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.499204] gitaly[918]: time="2026-09-13T10:38:03.781Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:38:13.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:03.780" grpc.time_ms=0.465 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.577707] gitaly[918]: time="2026-09-13T10:38:03.882Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:38:13.111" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:03.882" pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.600066] gitaly[918]: time="2026-09-13T10:38:03.884Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:38:13.111" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:03.882" grpc.time_ms=2.027 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.636234] gitaly[918]: time="2026-09-13T10:38:03.940Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 diskcache=c81fdae3-a547-41c9-8ee1-b04942cdfd01 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T10:38:13.122" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:03.893" pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.657689] gitaly[918]: time="2026-09-13T10:38:03.941Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1583 command.oublock=32 command.real_time_ms=23 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T10:38:13.122" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:03.893" grpc.time_ms=47.953 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.684640] gitaly[918]: time="2026-09-13T10:38:03.989Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:38:13.218" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:03.988" pid=918 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 196.685757] gitaly[918]: time="2026-09-13T10:38:03.989Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:38:13.218" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:03.988" grpc.time_ms=0.575 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.716223] gitaly[918]: time="2026-09-13T10:38:04.020Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:38:14.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:04.012" grpc.time_ms=8.349 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.782239] gitaly[918]: time="2026-09-13T10:38:04.086Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:38:14.076" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:04.086" pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.801694] gitaly[918]: time="2026-09-13T10:38:04.086Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:38:14.076" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:04.086" grpc.time_ms=0.245 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.833370] gitaly[918]: time="2026-09-13T10:38:04.138Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 diskcache=551be4e4-56d5-40f0-9de5-72b44430344e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T10:38:14.081" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:04.091" pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.844839] gitaly[918]: time="2026-09-13T10:38:04.149Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1584 command.oublock=32 command.real_time_ms=23 command.spawn_token_fork_ms=1 command.system_time_ms=15 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T10:38:14.081" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:04.091" grpc.time_ms=46.411 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.862863] gitaly[918]: time="2026-09-13T10:38:04.167Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:38:14.157" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:04.167" pid=918 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 196.872980] gitaly[918]: time="2026-09-13T10:38:04.167Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:38:14.157" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:04.167" grpc.time_ms=0.364 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.894892] gitaly[918]: time="2026-09-13T10:38:04.199Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:38:14.181" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:04.192" grpc.time_ms=6.946 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.931951] gitaly[918]: time="2026-09-13T10:38:04.236Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:38:14.225" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:04.236" grpc.time_ms=0.201 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 196.947655] gitaly[918]: time="2026-09-13T10:38:04.252Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T10:38:14.241" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:04.251" grpc.time_ms=0.418 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.365982] gitaly[918]: time="2026-09-13T10:38:04.670Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5GWZ946229NTPAEAW8V43 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:38:14.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:04.659" grpc.time_ms=10.931 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 197.781337] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:05 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 1843 gitlab # [ 197.783763] setup-gitlab-start[2511]: {"id":1,"message":null,"error":null} gitlab # [ 198.636770] sidekiq-cluster[2498]: {"severity":"INFO","time":"2026-09-13T10:38:05.924Z","message":"Starting cluster with 1 processes"} gitlab # [ 202.916548] gitaly[918]: time="2026-09-13T10:38:10.220Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H3P575DA5HVYTP7X5XSG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:38:20.198" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:10.209" grpc.time_ms=10.728 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 202.953502] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:10 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 139 gitlab # [ 202.956186] setup-gitlab-start[2212]: + mkdir -p /tmp/test-flake gitlab # [ 202.959130] setup-gitlab-start[2212]: + cd /tmp/test-flake gitlab # [ 202.960301] setup-gitlab-start[2212]: + git init -b master gitlab # [ 202.980989] setup-gitlab-start[2583]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 202.982307] setup-gitlab-start[2212]: + git config user.name test gitlab # [ 202.985658] setup-gitlab-start[2212]: + git config user.email test@example.com gitlab # [ 202.988192] setup-gitlab-start[2212]: + cat gitlab # [ 202.990119] setup-gitlab-start[2212]: + git add flake.nix gitlab # [ 202.992565] setup-gitlab-start[2212]: + git commit -m 'initial commit' gitlab # [ 203.008753] setup-gitlab-start[2588]: [master (root-commit) 2926555] initial commit gitlab # [ 203.009833] setup-gitlab-start[2588]: 1 file changed, 10 insertions(+) gitlab # [ 203.010969] setup-gitlab-start[2588]: create mode 100644 flake.nix gitlab # [ 203.012199] setup-gitlab-start[2212]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 203.017193] setup-gitlab-start[2212]: + git push -u origin master gitlab # [ 203.072451] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:10 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 27 gitlab # [ 203.164061] gitaly[918]: time="2026-09-13T10:38:10.468Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H3YA68F34BH3EZ22VXHS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:10.458" grpc.time_ms=10.156 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 203.193966] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:10 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 119 gitlab # [ 203.409445] gitaly[918]: time="2026-09-13T10:38:10.713Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:38:20.049" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:10.699" grpc.time_ms=13.505 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.450158] gitaly[918]: time="2026-09-13T10:38:10.754Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-13T10:38:40.043" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-13T10:38:10.743" grpc.time_ms=10.646 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.465172] gitaly[918]: time="2026-09-13T10:38:10.769Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M2D5H42KY41M09SFZRR7F1XF duration_ms=121ns method=POST pid=918 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 203.489572] gitaly[918]: time="2026-09-13T10:38:10.794Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M2D5H42KY41M09SFZRR7F1XF duration_ms=21ns method=POST pid=918 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 203.491378] gitaly[918]: time="2026-09-13T10:38:10.795Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5309 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:10.647" grpc.time_ms=148.239 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 203.539700] gitaly[918]: time="2026-09-13T10:38:10.844Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4779 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:10.843" grpc.time_ms=0.513 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 203.587642] gitaly[918]: time="2026-09-13T10:38:10.892Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4778 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:10.891" grpc.time_ms=0.534 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 203.625545] gitaly[918]: time="2026-09-13T10:38:10.930Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:10.928" grpc.time_ms=1.344 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 203.702563] gitaly[918]: time="2026-09-13T10:38:11.007Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:38:21.100" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:11.001" grpc.time_ms=5.721 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.744941] gitaly[918]: time="2026-09-13T10:38:11.049Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1564 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-13T10:38:21.021" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:11.031" grpc.time_ms=17.811 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.764350] gitaly[918]: time="2026-09-13T10:38:11.069Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M2D5H42KY41M09SFZRR7F1XF duration_ms=108ns method=POST pid=918 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 203.767431] gitaly[918]: time="2026-09-13T10:38:11.069Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4803 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:10.959" grpc.time_ms=110.204 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 203.785760] gitaly[918]: time="2026-09-13T10:38:11.090Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF diskcache=e5649214-4244-4022-bc2c-55d6e3924c47 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:10.571" pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 203.795102] gitaly[918]: time="2026-09-13T10:38:11.099Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=233 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13892 command.oublock=32 command.real_time_ms=511 command.spawn_token_fork_ms=1 command.system_time_ms=148 command.user_time_ms=85 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:10.571" grpc.time_ms=519.311 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 203.796194] gitlab-workhorse[829]: time="2026-09-13T10:38:11Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 203.796845] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:11 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 584 gitlab # [ 203.799422] setup-gitlab-start[2592]: To http://gitlab/root/test-flake.git gitlab # [ 203.800158] setup-gitlab-start[2592]: * [new branch] master -> master gitlab # [ 203.815871] setup-gitlab-start[2592]: branch 'master' set up to track 'origin/master'. gitlab # [ 203.818766] systemd[1]: Finished setup-gitlab.service. gitlab # [ 203.821392] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 205.11 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 205.001977] nixbot[2669]: {"ts": "2026-09-13T10:38:12.305Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 205.035127] nixbot[2669]: {"ts": "2026-09-13T10:38:12.339Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 205.064648] nixbot[2669]: {"ts": "2026-09-13T10:38:12.369Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 205.073430] nixbot[2669]: {"ts": "2026-09-13T10:38:12.377Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 205.081159] nixbot[2669]: {"ts": "2026-09-13T10:38:12.385Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 205.093413] nixbot[2669]: {"ts": "2026-09-13T10:38:12.397Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 205.101830] nixbot[2669]: {"ts": "2026-09-13T10:38:12.406Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 205.108309] nixbot[2669]: {"ts": "2026-09-13T10:38:12.412Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 205.120201] nixbot[2669]: {"ts": "2026-09-13T10:38:12.424Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 205.127803] nixbot[2669]: {"ts": "2026-09-13T10:38:12.432Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 205.138653] nixbot[2669]: {"ts": "2026-09-13T10:38:12.443Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 205.150860] nixbot[2669]: {"ts": "2026-09-13T10:38:12.455Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 205.155755] nixbot[2669]: {"ts": "2026-09-13T10:38:12.460Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 205.163557] nixbot[2669]: {"ts": "2026-09-13T10:38:12.468Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 205.169146] nixbot[2669]: {"ts": "2026-09-13T10:38:12.473Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 205.176109] nixbot[2669]: {"ts": "2026-09-13T10:38:12.480Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 205.182952] nixbot[2669]: {"ts": "2026-09-13T10:38:12.487Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 205.186053] nixbot[2669]: {"ts": "2026-09-13T10:38:12.490Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 205.198119] nixbot[2669]: {"ts": "2026-09-13T10:38:12.502Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 205.204740] nixbot[2669]: {"ts": "2026-09-13T10:38:12.509Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 205.210979] nixbot[2669]: {"ts": "2026-09-13T10:38:12.515Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 205.221609] nixbot[2669]: {"ts": "2026-09-13T10:38:12.526Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 205.227969] nixbot[2669]: {"ts": "2026-09-13T10:38:12.532Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 205.232334] nixbot[2669]: {"ts": "2026-09-13T10:38:12.536Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 205.238357] nixbot[2669]: {"ts": "2026-09-13T10:38:12.542Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 205.243990] nixbot[2669]: {"ts": "2026-09-13T10:38:12.548Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 205.249352] nixbot[2669]: {"ts": "2026-09-13T10:38:12.553Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 205.253508] nixbot[2669]: {"ts": "2026-09-13T10:38:12.558Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 205.258366] nixbot[2669]: {"ts": "2026-09-13T10:38:12.562Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 205.269873] nixbot[2669]: {"ts": "2026-09-13T10:38:12.574Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 205.276062] nixbot[2669]: {"ts": "2026-09-13T10:38:12.580Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 205.280215] nixbot[2669]: {"ts": "2026-09-13T10:38:12.584Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 205.284320] nixbot[2669]: {"ts": "2026-09-13T10:38:12.588Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} gitlab # [ 205.291699] nixbot[2669]: {"ts": "2026-09-13T10:38:12.596Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"} gitlab # [ 205.296112] nixbot[2669]: {"ts": "2026-09-13T10:38:12.600Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"} gitlab # [ 205.648980] nixbot[2669]: INFO: Started server process [2669] gitlab # [ 205.651327] nixbot[2669]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 205.670450] nixbot[2669]: INFO: ::ffff:127.0.0.1:43904 - "GET /health HTTP/1.1" 200 OK gitlab # [ 205.680273] systemd[1]: Started nixbot CI service. gitlab # [ 205.683614] systemd[1]: Reached target Multi-User System. gitlab # [ 205.686561] systemd[1]: Startup finished in 1.239s (kernel) + 3.629s (initrd) + 3min 20.817s (userspace) = 3min 25.685s. gitlab # [ 206.546294] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:13 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 877 gitlab # [ 206.548247] nixbot[2669]: {"ts": "2026-09-13T10:38:13.852Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 206.576554] nixbot[2669]: {"ts": "2026-09-13T10:38:13.881Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab: (finished: waiting for unit nixbot.service, in 2.09 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 206.640223] nixbot[2669]: INFO: ::ffff:127.0.0.1:43912 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.02 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 206.671955] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 80 gitlab # [ 206.674180] nixbot[2669]: {"ts": "2026-09-13T10:38:13.978Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 206.676354] nixbot[2669]: {"ts": "2026-09-13T10:38:13.979Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 206.720288] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 52 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.08 seconds) gitlab # [ 206.827258] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:14 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 150 gitlab # [ 206.829346] nixbot[2669]: {"ts": "2026-09-13T10:38:14.133Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab # [ 206.949368] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:14 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 111 gitlab # [ 206.950846] nixbot[2669]: {"ts": "2026-09-13T10:38:14.255Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 207.030153] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 67 gitlab # [ 207.033708] nixbot[2669]: {"ts": "2026-09-13T10:38:14.338Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 207.126513] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:14 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 87 gitlab # [ 207.132497] nixbot[2669]: {"ts": "2026-09-13T10:38:14.437Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 207.230177] gitaly[918]: time="2026-09-13T10:38:14.534Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H7XGREXY8Y21ZHC7V4D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-13T10:38:44.016" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:14.516" grpc.time_ms=16.499 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.262104] gitaly[918]: time="2026-09-13T10:38:14.566Z" level=info msg="finished unary call with code OK" catfile.duration_ms=17 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=17 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H7XGREXY8Y21ZHC7V4D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:38:44.047" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:14.548" grpc.time_ms=18.304 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.281418] gitaly[918]: time="2026-09-13T10:38:14.586Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H7XGREXY8Y21ZHC7V4D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-13T10:38:24.002" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:14.574" grpc.time_ms=10.894 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 207.308914] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 164 gitlab # [ 207.310104] nixbot[2669]: {"ts": "2026-09-13T10:38:14.614Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 207.420231] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 105 gitlab # [ 207.422318] nixbot[2669]: {"ts": "2026-09-13T10:38:14.725Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 207.433826] nixbot[2669]: {"ts": "2026-09-13T10:38:14.738Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 207.532332] gitaly[918]: time="2026-09-13T10:38:14.835Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H883ZVTNEA0AHRXP013J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:14.822" grpc.time_ms=12.991 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 207.536792] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:14 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 52 gitlab # [ 207.655391] gitaly[918]: time="2026-09-13T10:38:14.960Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H8AP4NPGCW40V2X4PCN2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:14.946" pid=918 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 207.657916] gitaly[918]: time="2026-09-13T10:38:14.960Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H8AP4NPGCW40V2X4PCN2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:14.946" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 207.660356] gitaly[918]: time="2026-09-13T10:38:14.960Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H8AP4NPGCW40V2X4PCN2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:14.946" grpc.time_ms=13.999 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 207.710264] gitlab-workhorse[829]: time="2026-09-13T10:38:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 207.711320] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 143 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 207.770457] gitaly[918]: time="2026-09-13T10:38:15.075Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H8FC5HTJ6927VQQSF9VV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:15.057" pid=918 remote_ip="2001:db8:1::1" request_sha=3f891105f5cf867e3fc25dfadf22e40f24a456c1 response_bytes=228 span.kind=server system=grpc gitlab # [ 207.771653] gitaly[918]: time="2026-09-13T10:38:15.075Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1205 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H8FC5HTJ6927VQQSF9VV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:15.057" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 207.772656] gitlab-workhorse[829]: time="2026-09-13T10:38:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=228 gitlab # [ 207.784188] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 228 "" "git/2.55.0" 55 gitlab # [ 207.784936] gitaly[918]: time="2026-09-13T10:38:15.075Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1205 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H8FC5HTJ6927VQQSF9VV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:15.057" grpc.time_ms=18.308 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 207.829266] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 70 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.11 seconds) (finished: subtest: project discovered and webhook registered, in 1.19 seconds) subtest: push delivers the webhook gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' gitlab # [ 207.969358] gitaly[918]: time="2026-09-13T10:38:15.273Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H8K73HCWEJKYK17V7A17 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:15.261" grpc.time_ms=12.241 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 207.971456] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 131 gitlab # [ 208.029734] gitaly[918]: time="2026-09-13T10:38:15.334Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H8QFBMY46BE8ACVJ81G2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:15.321" pid=918 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 208.030886] gitaly[918]: time="2026-09-13T10:38:15.334Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H8QFBMY46BE8ACVJ81G2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:15.321" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 208.031710] gitaly[918]: time="2026-09-13T10:38:15.334Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H8QFBMY46BE8ACVJ81G2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:15.321" grpc.time_ms=13.09 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 208.074489] gitlab-workhorse[829]: time="2026-09-13T10:38:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 208.075263] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 99 gitlab # [ 208.175067] gitaly[918]: time="2026-09-13T10:38:15.479Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H8W1F9WM1D9VNS4KJWJD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:15.467" grpc.time_ms=12.448 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 208.176092] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:15 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 54 gitlab # [ 208.306671] gitaly[918]: time="2026-09-13T10:38:15.611Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H8YRF2QGES7TXK0GA4FF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:15.589" pid=918 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 208.307942] gitaly[918]: time="2026-09-13T10:38:15.611Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1167 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H8YRF2QGES7TXK0GA4FF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:15.589" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 208.354878] gitaly[918]: time="2026-09-13T10:38:15.659Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1167 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H8YRF2QGES7TXK0GA4FF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:15.589" grpc.time_ms=69.77 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 208.372351] gitlab-workhorse[829]: time="2026-09-13T10:38:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 208.374213] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:15 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 147 gitlab # [ 208.459473] gitaly[918]: time="2026-09-13T10:38:15.764Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:38:45.262" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:15.763" grpc.time_ms=0.812 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 208.492678] gitaly[918]: time="2026-09-13T10:38:15.797Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-13T10:38:45.289" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:15.790" grpc.time_ms=7.073 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 208.516477] gitaly[918]: time="2026-09-13T10:38:15.816Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:38:25.238" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:15.809" grpc.time_ms=7.326 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 208.527969] gitaly[918]: time="2026-09-13T10:38:15.832Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:38:25.255" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:15.825" grpc.time_ms=7.05 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 208.588717] gitaly[918]: time="2026-09-13T10:38:15.893Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T10:38:25.241" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:15.892" grpc.time_ms=0.553 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 208.626917] gitaly[918]: time="2026-09-13T10:38:15.931Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T10:38:25.270" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:15.921" grpc.time_ms=9.894 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 208.687527] gitaly[918]: time="2026-09-13T10:38:15.992Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:15.990" grpc.time_ms=1.176 pid=918 span.kind=server system=grpc gitlab # [ 208.716943] gitaly[918]: time="2026-09-13T10:38:16.021Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:16.021" grpc.time_ms=0.337 pid=918 span.kind=server system=grpc gitlab # [ 208.733297] gitaly[918]: time="2026-09-13T10:38:16.037Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA diskcache=6ddb8568-ef75-4209-8e95-4a6348cdc698 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:38:25.002" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:15.953" pid=918 ref=refs/keep-around/2926555362c994dd725f8b6ab88adcd6b3aca0df remote_ip="2001:db8:1::1" revision=2926555362c994dd725f8b6ab88adcd6b3aca0df span.kind=server system=grpc username=root gitlab # [ 208.733866] gitaly[918]: time="2026-09-13T10:38:16.038Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=55 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5384 command.oublock=8 command.real_time_ms=78 command.spawn_token_fork_ms=0 command.system_time_ms=36 command.user_time_ms=19 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:38:25.002" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:15.953" grpc.time_ms=85.021 pid=918 ref=refs/keep-around/2926555362c994dd725f8b6ab88adcd6b3aca0df remote_ip="2001:db8:1::1" revision=2926555362c994dd725f8b6ab88adcd6b3aca0df span.kind=server system=grpc username=root gitlab # [ 208.738380] gitaly[918]: time="2026-09-13T10:38:16.043Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:38:46.042" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:16.042" grpc.time_ms=0.43 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 208.936545] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2926555362c994dd725f8b6ab88adcd6b3aca0df HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 522 gitlab # [ 208.940349] nixbot[2669]: {"ts": "2026-09-13T10:38:16.244Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2926555362c994dd725f8b6ab88adcd6b3aca0df \"HTTP/1.1 201 Created\""} gitlab # [ 208.951988] nixbot[2669]: {"ts": "2026-09-13T10:38:16.256Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-2926555362c9-0"} gitlab # [ 209.152796] nixbot[2669]: {"ts": "2026-09-13T10:38:16.457Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-2926555362c9-0", "argv0": "bwrap"} gitlab # [ 209.243126] systemd[1]: Started Nix Daemon. gitlab # [ 209.318395] nix-daemon[2835]: accepted connection from pid 2830, user nixbot gitlab # [ 213.449484] nix-daemon[2835]: accepted connection from pid 2877, user nixbot gitlab # [ 213.467762] nix-daemon[2835]: accepted connection from pid 2830, user nixbot gitlab # [ 218.295259] nix-daemon[2835]: accepted connection from pid 2830, user nixbot gitlab # [ 218.391523] nix-daemon[2835]: accepted connection from pid 2926, user nixbot gitlab # [ 218.430781] gitaly[918]: time="2026-09-13T10:38:25.734Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5HJWN42X1WNN29RJCKZBH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:38:55.233" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:25.733" grpc.time_ms=0.734 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.488708] gitaly[918]: time="2026-09-13T10:38:25.793Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2D5HJWN42X1WNN29RJCKZBH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:38:35.211" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:25.782" grpc.time_ms=11.101 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.560082] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:25 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2926555362c994dd725f8b6ab88adcd6b3aca0df HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 178 gitlab # [ 218.566680] nixbot[2669]: {"ts": "2026-09-13T10:38:25.870Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2926555362c994dd725f8b6ab88adcd6b3aca0df \"HTTP/1.1 201 Created\""} gitlab # [ 218.636933] gitaly[918]: time="2026-09-13T10:38:25.941Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5HK2TF17VMXT95WBVG669 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:38:55.040" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:25.940" grpc.time_ms=0.842 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.727338] gitaly[918]: time="2026-09-13T10:38:26.032Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5HK2TF17VMXT95WBVG669 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:38:36.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:26.023" grpc.time_ms=7.828 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.818186] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2926555362c994dd725f8b6ab88adcd6b3aca0df HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 240 gitlab # [ 218.819543] nixbot[2669]: {"ts": "2026-09-13T10:38:26.124Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2926555362c994dd725f8b6ab88adcd6b3aca0df \"HTTP/1.1 201 Created\""} gitlab # [ 223.221881] sidekiq-cluster[2556]: 2026-09-13T10:38:30.525Z pid=2556 tid=xuc INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2556, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>439783424} gitlab # [ 223.423743] sidekiq-cluster[2556]: W, [2026-09-13T10:38:30.728363 #2556] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 223.471676] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:30.776Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 223.633099] nix-daemon[2835]: accepted connection from pid 3010, user nixbot gitlab # [ 223.719360] gitaly[918]: time="2026-09-13T10:38:31.023Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5HR1JKY40ER7T40364Y1X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:01.021" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:31.022" grpc.time_ms=0.676 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 223.743455] gitaly[918]: time="2026-09-13T10:38:31.048Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5HR1JKY40ER7T40364Y1X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:38:41.021" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:31.042" grpc.time_ms=5.898 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 223.796217] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2926555362c994dd725f8b6ab88adcd6b3aca0df HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 137 gitlab # [ 223.797217] nixbot[2669]: {"ts": "2026-09-13T10:38:31.101Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2926555362c994dd725f8b6ab88adcd6b3aca0df \"HTTP/1.1 201 Created\""} gitlab # [ 224.072101] gitaly[918]: time="2026-09-13T10:38:31.376Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5HRCSTHMXG0N9G8MXJ84C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:31.369" grpc.time_ms=6.865 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 224.097389] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 80 gitlab # [ 224.167046] gitaly[918]: time="2026-09-13T10:38:31.471Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5HRFERTFFZD89Y9YY88AM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:31.460" pid=918 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 224.167850] gitaly[918]: time="2026-09-13T10:38:31.471Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5HRFERTFFZD89Y9YY88AM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:31.460" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 224.168699] gitlab-workhorse[829]: time="2026-09-13T10:38:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 224.170099] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:31 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 65 gitlab # [ 224.171775] gitaly[918]: time="2026-09-13T10:38:31.471Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5HRFERTFFZD89Y9YY88AM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:31.460" grpc.time_ms=11.288 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 224.272513] gitaly[918]: time="2026-09-13T10:38:31.577Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5HRKG45R9GKA7HR0435SE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:31.567" grpc.time_ms=9.164 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 224.274607] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 43 gitlab # [ 224.319436] gitaly[918]: time="2026-09-13T10:38:31.624Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5HRMX0NRJ8V2JR4F2023C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:31.617" pid=918 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 224.329208] gitaly[918]: time="2026-09-13T10:38:31.624Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5HRMX0NRJ8V2JR4F2023C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:31.617" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 224.341121] gitlab-workhorse[829]: time="2026-09-13T10:38:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 224.342942] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:31 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 43 gitlab # [ 224.345098] gitaly[918]: time="2026-09-13T10:38:31.624Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5HRMX0NRJ8V2JR4F2023C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:31.617" grpc.time_ms=7.231 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 225.354825] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:32.658Z","message":"GitLab reliable fetch activated!"} gitlab # [ 225.356710] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:32.658Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 245.600615] postgres[3239]: [3239] ERROR: relation "table_without_models" does not exist gitlab # [ 245.602596] postgres[3239]: [3239] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 248.108754] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:55.412Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 248.111830] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:55.416Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 248.113918] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:55.416Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 248.115426] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:55.416Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 248.116920] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:55.418Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 40.32 seconds) gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 248.250040] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:55 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 38 gitlab # [ 248.435434] gitaly[918]: time="2026-09-13T10:38:55.739Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG28D6A20JZV11DH7BT7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:55.726" grpc.time_ms=12.597 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 248.463880] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:55 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 178 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 137.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 248.687870] gitaly[918]: time="2026-09-13T10:38:55.992Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-13T10:39:25.479" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-13T10:38:55.979" grpc.time_ms=12.138 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.719217] gitaly[918]: time="2026-09-13T10:38:56.022Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE duration_ms=88ns method=POST pid=918 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 248.738361] gitaly[918]: time="2026-09-13T10:38:56.043Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE duration_ms=19ns method=POST pid=918 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 248.744496] gitaly[918]: time="2026-09-13T10:38:56.043Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5309 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:55.932" grpc.time_ms=110.623 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 248.793097] gitaly[918]: time="2026-09-13T10:38:56.097Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4779 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:56.095" grpc.time_ms=1.484 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 248.835107] gitaly[918]: time="2026-09-13T10:38:56.139Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4778 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:56.138" grpc.time_ms=1.39 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 248.871993] gitaly[918]: time="2026-09-13T10:38:56.176Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:56.176" grpc.time_ms=0.394 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 248.937441] gitaly[918]: time="2026-09-13T10:38:56.242Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE duration_ms=34ns method=POST pid=918 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 248.939438] gitaly[918]: time="2026-09-13T10:38:56.242Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4803 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:56.206" grpc.time_ms=35.63 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 248.969217] gitaly[918]: time="2026-09-13T10:38:56.273Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE diskcache=d50ef0aa-c401-4c82-a55f-d55bb693320d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:55.859" pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 248.969899] gitaly[918]: time="2026-09-13T10:38:56.274Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=231 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13517 command.oublock=32 command.real_time_ms=407 command.spawn_token_fork_ms=0 command.system_time_ms=167 command.user_time_ms=64 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:38:55.859" grpc.time_ms=415.041 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 248.970882] gitlab-workhorse[829]: time="2026-09-13T10:38:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 248.971674] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:38:56 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 480 gitlab # To http://gitlab/root/test-flake.git gitlab # 2926555..d2fb939 master -> master gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.85 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846' gitlab # [ 249.067716] nixbot[2669]: INFO: ::ffff:127.0.0.1:56562 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846', in 0.06 seconds) gitlab # [ 249.804540] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:57.108Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"9e6a3020880e6101553be238","created_at":"2026-09-13T10:38:04.041Z","trace_propagation_headers":{"sentry-trace":"1b74203e81e0425291188cc92c057df5-63c4cd5067d049f1","baggage":"sentry-trace_id=1b74203e81e0425291188cc92c057df5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5GWZ946229NTPAEAW8V43","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:04.068Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-9e6a3020880e6101553be238: start","job_status":"start","queue_duration_s":53.039354,"scheduling_latency_s":53.039409} gitlab # [ 249.996545] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:57.300Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"9e6a3020880e6101553be238","created_at":"2026-09-13T10:38:04.041Z","trace_propagation_headers":{"sentry-trace":"1b74203e81e0425291188cc92c057df5-63c4cd5067d049f1","baggage":"sentry-trace_id=1b74203e81e0425291188cc92c057df5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5GWZ946229NTPAEAW8V43","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:04.068Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-9e6a3020880e6101553be238: done: 0.192935 sec","job_status":"done","queue_duration_s":53.039354,"scheduling_latency_s":53.039409,"redis_calls":24,"redis_duration_s":0.006166,"redis_read_bytes":1038,"redis_write_bytes":2604,"redis_feature_flag_calls":14,"redis_feature_flag_duration_s":0.003326,"redis_feature_flag_read_bytes":1030,"redis_feature_flag_write_bytes":1439,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001417,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001423,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.062363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.090549218,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.192935,"completed_at":"2026-09-13T10:38:57.300Z","load_balancing_strategy":"primary","db_duration_s":0.064048,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846' gitlab # [ 249.999245] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:57.302Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"ac6d8cc24933524357e86d8f","created_at":"2026-09-13T10:38:04.078Z","trace_propagation_headers":{"sentry-trace":"1b74203e81e0425291188cc92c057df5-63c4cd5067d049f1","baggage":"sentry-trace_id=1b74203e81e0425291188cc92c057df5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5GWZ946229NTPAEAW8V43","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:04.082Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-ac6d8cc24933524357e86d8f: start","job_status":"start","queue_duration_s":53.219985,"scheduling_latency_s":53.220001} gitlab # [ 250.079190] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:57.383Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"ac6d8cc24933524357e86d8f","created_at":"2026-09-13T10:38:04.078Z","trace_propagation_headers":{"sentry-trace":"1b74203e81e0425291188cc92c057df5-63c4cd5067d049f1","baggage":"sentry-trace_id=1b74203e81e0425291188cc92c057df5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5GWZ946229NTPAEAW8V43","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:04.082Z","job_size_bytes":2,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-ac6d8cc24933524357e86d8f: done: 0.081176 sec","job_status":"done","queue_duration_s":53.219985,"scheduling_latency_s":53.220001,"redis_calls":14,"redis_duration_s":0.003464,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000675,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001604,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001185,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.06305913199999999,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.081176,"completed_at":"2026-09-13T10:38:57.383Z","load_balancing_strategy":"primary","db_duration_s":0.023458,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 250.081100] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:57.385Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"62e4ecb9f25c080682874686","created_at":"2026-09-13T10:38:04.578Z","trace_propagation_headers":{"sentry-trace":"1b74203e81e0425291188cc92c057df5-63c4cd5067d049f1","baggage":"sentry-trace_id=1b74203e81e0425291188cc92c057df5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5GWZ946229NTPAEAW8V43","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:04.584Z","job_size_bytes":5,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-62e4ecb9f25c080682874686: start","job_status":"start","queue_duration_s":52.800712,"scheduling_latency_s":52.800728} gitlab # [ 250.158972] nixbot[2669]: INFO: ::ffff:127.0.0.1:56576 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846', in 0.09 seconds) gitlab # [ 250.241633] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:57.545Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"62e4ecb9f25c080682874686","created_at":"2026-09-13T10:38:04.578Z","trace_propagation_headers":{"sentry-trace":"1b74203e81e0425291188cc92c057df5-63c4cd5067d049f1","baggage":"sentry-trace_id=1b74203e81e0425291188cc92c057df5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5GWZ946229NTPAEAW8V43","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:04.584Z","job_size_bytes":5,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-62e4ecb9f25c080682874686: done: 0.159788 sec","job_status":"done","queue_duration_s":52.800712,"scheduling_latency_s":52.800728,"redis_calls":12,"redis_duration_s":0.003157,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000599,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002131,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.118272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00045732,"duration_s":0.159788,"completed_at":"2026-09-13T10:38:57.545Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029999,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 250.244578] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:57.547Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"31fc7868a7ac4770f68f90fc","created_at":"2026-09-13T10:38:04.606Z","trace_propagation_headers":{"sentry-trace":"1b74203e81e0425291188cc92c057df5-63c4cd5067d049f1","baggage":"sentry-trace_id=1b74203e81e0425291188cc92c057df5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5GWZ946229NTPAEAW8V43","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:04.610Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-31fc7868a7ac4770f68f90fc: start","job_status":"start","queue_duration_s":52.937536,"scheduling_latency_s":52.937569} gitlab # [ 250.431198] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:57.735Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"31fc7868a7ac4770f68f90fc","created_at":"2026-09-13T10:38:04.606Z","trace_propagation_headers":{"sentry-trace":"1b74203e81e0425291188cc92c057df5-63c4cd5067d049f1","baggage":"sentry-trace_id=1b74203e81e0425291188cc92c057df5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5GWZ946229NTPAEAW8V43","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:04.610Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-31fc7868a7ac4770f68f90fc: done: 0.187609 sec","job_status":"done","queue_duration_s":52.937536,"scheduling_latency_s":52.937569,"redis_calls":11,"redis_duration_s":0.002582,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000626,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001484,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.145,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.145,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.110803,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000332445,"duration_s":0.187609,"completed_at":"2026-09-13T10:38:57.735Z","load_balancing_strategy":"primary","db_duration_s":0.074373,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.433822] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:57.736Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"cde02df10af1765ff16861fa","created_at":"2026-09-13T10:38:04.612Z","trace_propagation_headers":{"sentry-trace":"1b74203e81e0425291188cc92c057df5-63c4cd5067d049f1","baggage":"sentry-trace_id=1b74203e81e0425291188cc92c057df5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5GWZ946229NTPAEAW8V43","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:04.615Z","job_size_bytes":89,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-cde02df10af1765ff16861fa: start","job_status":"start","queue_duration_s":53.120809,"scheduling_latency_s":53.120824} gitlab # [ 250.513209] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:57.764Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"cde02df10af1765ff16861fa","created_at":"2026-09-13T10:38:04.612Z","trace_propagation_headers":{"sentry-trace":"1b74203e81e0425291188cc92c057df5-63c4cd5067d049f1","baggage":"sentry-trace_id=1b74203e81e0425291188cc92c057df5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5GWZ946229NTPAEAW8V43","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:04.615Z","job_size_bytes":89,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-cde02df10af1765ff16861fa: done: 0.027445 sec","job_status":"done","queue_duration_s":53.120809,"scheduling_latency_s":53.120824,"redis_calls":11,"redis_duration_s":0.002423,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000707,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001288,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000329931,"duration_s":0.027445,"completed_at":"2026-09-13T10:38:57.764Z","load_balancing_strategy":"primary","db_duration_s":0.005326,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.520381] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:57.765Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 2926555362c994dd725f8b6ab88adcd6b3aca0df refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"5b9ccef609c932f96f18241f","created_at":"2026-09-13T10:38:10.985Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-6127bdc7c7034cec","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:38:10.989Z","job_size_bytes":185,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-5b9ccef609c932f96f18241f: start","job_status":"start","queue_duration_s":46.776153,"scheduling_latency_s":46.776168} gitlab # [ 250.536259] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:57.776Z","message":"changes: \"0000000000000000000000000000000000000000 2926555362c994dd725f8b6ab88adcd6b3aca0df refs/heads/master\\n\""} gitlab # [ 250.539190] gitaly[918]: time="2026-09-13T10:38:57.808Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:39:07.011" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:57.801" grpc.time_ms=7.142 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.586793] gitaly[918]: time="2026-09-13T10:38:57.823Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:39:07.022" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:57.813" grpc.time_ms=10.239 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.587687] gitaly[918]: time="2026-09-13T10:38:57.878Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-13T10:39:07.078" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:57.868" grpc.time_ms=10.079 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.617089] gitaly[918]: time="2026-09-13T10:38:57.921Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3931 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:57.921" grpc.time_ms=0.392 pid=918 span.kind=server system=grpc gitlab # [ 250.652310] gitaly[918]: time="2026-09-13T10:38:57.957Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:57.956" grpc.time_ms=0.385 pid=918 span.kind=server system=grpc gitlab # [ 250.670570] gitaly[918]: time="2026-09-13T10:38:57.975Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF diskcache=5727e4d8-1e19-4f7d-b1ce-fb325546029f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:39:07.099" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:57.890" pid=918 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 250.680348] gitaly[918]: time="2026-09-13T10:38:57.984Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=74 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5084 command.oublock=8 command.real_time_ms=79 command.spawn_token_fork_ms=0 command.system_time_ms=46 command.user_time_ms=28 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:39:07.099" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:57.890" grpc.time_ms=85.305 pid=918 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 250.699185] gitaly[918]: time="2026-09-13T10:38:58.003Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:39:08.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:58.003" grpc.time_ms=0.196 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.719021] gitaly[918]: time="2026-09-13T10:38:58.023Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=938 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-13T10:39:08.009" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:58.009" grpc.time_ms=13.697 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.817666] gitaly[918]: time="2026-09-13T10:38:58.122Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=341 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-13T10:39:28.014" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:58.115" grpc.time_ms=7.118 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.843669] gitaly[918]: time="2026-09-13T10:38:58.148Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T10:39:08.147" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:58.147" grpc.time_ms=0.423 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.854771] gitaly[918]: time="2026-09-13T10:38:58.159Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-13T10:39:08.151" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-13T10:38:58.151" grpc.time_ms=7.427 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.890328] gitaly[918]: time="2026-09-13T10:38:58.194Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:39:08.185" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:58.186" grpc.time_ms=8.547 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.903366] gitaly[918]: time="2026-09-13T10:38:58.204Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:39:08.198" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:58.199" grpc.time_ms=5.611 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.924188] gitaly[918]: time="2026-09-13T10:38:58.228Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=960 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-13T10:39:28.113" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:58.213" grpc.time_ms=14.943 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846' gitlab # [ 251.208837] nixbot[2669]: INFO: ::ffff:127.0.0.1:56586 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846', in 0.05 seconds) gitlab # [ 251.318993] gitaly[918]: time="2026-09-13T10:38:58.623Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-13T10:39:08.001" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:58.622" grpc.time_ms=1.222 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.352554] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:58.656Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 2926555362c994dd725f8b6ab88adcd6b3aca0df refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"5b9ccef609c932f96f18241f","created_at":"2026-09-13T10:38:10.985Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-6127bdc7c7034cec","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:38:10.989Z","job_size_bytes":185,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-5b9ccef609c932f96f18241f: done: 0.890686 sec","job_status":"done","queue_duration_s":46.776153,"scheduling_latency_s":46.776168,"gitaly_calls":13,"gitaly_duration_s":0.25039,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.045975,"redis_read_bytes":2103,"redis_write_bytes":21751,"redis_cache_calls":15,"redis_cache_duration_s":0.004264,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.00658,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001615,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7839,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004662,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.01685,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.012004,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.196,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.22,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.059,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.510152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000441956,"exclusive_lock_hold_duration_s":0.02524399,"duration_s":0.890686,"completed_at":"2026-09-13T10:38:58.656Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.102288,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 251.408234] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:58.657Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"9cadf07d2a240d9924166a1b","created_at":"2026-09-13T10:38:16.078Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2abc941a6bea4e23","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:16.086Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-9cadf07d2a240d9924166a1b: start","job_status":"start","queue_duration_s":42.570976,"scheduling_latency_s":42.570994} gitlab # [ 251.419072] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:58.723Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"9cadf07d2a240d9924166a1b","created_at":"2026-09-13T10:38:16.078Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2abc941a6bea4e23","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:16.086Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-9cadf07d2a240d9924166a1b: done: 0.065761 sec","job_status":"done","queue_duration_s":42.570976,"scheduling_latency_s":42.570994,"redis_calls":16,"redis_duration_s":0.004424,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002561,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001442,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000421,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000308978,"duration_s":0.065761,"completed_at":"2026-09-13T10:38:58.723Z","load_balancing_strategy":"primary","db_duration_s":0.021588,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 251.426954] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:58.725Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f13f3ed896af005f41b690df","created_at":"2026-09-13T10:38:16.087Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2abc941a6bea4e23","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:16.094Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f13f3ed896af005f41b690df: start","job_status":"start","queue_duration_s":42.630789,"scheduling_latency_s":42.630806} gitlab # [ 251.527455] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:58.831Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f13f3ed896af005f41b690df","created_at":"2026-09-13T10:38:16.087Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2abc941a6bea4e23","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:16.094Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f13f3ed896af005f41b690df: done: 0.106769 sec","job_status":"done","queue_duration_s":42.630789,"scheduling_latency_s":42.630806,"redis_calls":21,"redis_duration_s":0.005858,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002725,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001523,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00161,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000368483,"duration_s":0.106769,"completed_at":"2026-09-13T10:38:58.831Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029325,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.569549] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:58.874Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a1d01b99b4dc37685259013f","created_at":"2026-09-13T10:38:16.206Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2abc941a6bea4e23","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:16.217Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a1d01b99b4dc37685259013f: start","job_status":"start","queue_duration_s":42.65683,"scheduling_latency_s":42.656853} gitlab # [ 251.776306] gitaly[918]: time="2026-09-13T10:38:59.080Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:59.079" grpc.time_ms=0.971 pid=918 span.kind=server system=grpc gitlab # [ 251.834055] gitaly[918]: time="2026-09-13T10:38:59.138Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:59.137" grpc.time_ms=0.582 pid=918 span.kind=server system=grpc gitlab # [ 251.889446] gitaly[918]: time="2026-09-13T10:38:59.193Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:38:59.193" grpc.time_ms=0.648 pid=918 span.kind=server system=grpc gitlab # [ 251.911319] gitaly[918]: time="2026-09-13T10:38:59.216Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA diskcache=7feaca70-9857-4a21-9955-1e7d5db9bcb7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:39:29.035" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:59.035" pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.920957] gitaly[918]: time="2026-09-13T10:38:59.225Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=164 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7594 command.oublock=0 command.real_time_ms=173 command.spawn_token_fork_ms=0 command.system_time_ms=99 command.user_time_ms=65 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:39:29.035" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:59.035" grpc.time_ms=180.147 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.126651] gitaly[918]: time="2026-09-13T10:38:59.431Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:39:29.329" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:59.429" grpc.time_ms=1.432 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.157117] gitaly[918]: time="2026-09-13T10:38:59.461Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H955G51KR0KEH017P1WA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:29.359" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:59.460" grpc.time_ms=1.065 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.186811] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:59.491Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a1d01b99b4dc37685259013f","created_at":"2026-09-13T10:38:16.206Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2abc941a6bea4e23","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:16.217Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2556,"job_status":"rescheduled","message":"PipelineProcessWorker JID-a1d01b99b4dc37685259013f: rescheduled"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846', in 0.04 seconds) gitlab # [ 252.215137] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:59.519Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a1d01b99b4dc37685259013f","created_at":"2026-09-13T10:38:16.206Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2abc941a6bea4e23","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:16.217Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a1d01b99b4dc37685259013f: done: 0.645525 sec","job_status":"done","queue_duration_s":42.65683,"scheduling_latency_s":42.656853,"gitaly_calls":3,"gitaly_duration_s":0.189948,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.073926,"redis_read_bytes":460,"redis_write_bytes":29292,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001541,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001161,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.011903,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.005573,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15580,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.027317,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000671,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":165,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00116,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.023059,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.14,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.024,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.028,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.108,"db_ci_duration_s":0.032,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.313767,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.531499674,"duration_s":0.645525,"completed_at":"2026-09-13T10:38:59.519Z","load_balancing_strategy":"primary","db_duration_s":0.152225,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 252.259235] nixbot[2669]: INFO: ::ffff:127.0.0.1:56598 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846 HTTP/1.1" 200 OK gitlab # [ 252.266809] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:59.571Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","2926555362c994dd725f8b6ab88adcd6b3aca0df d2fb939f2bad435ffeda072a310aa498ab167846 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"f3a3310beeacf9891f6c3d7b","created_at":"2026-09-13T10:38:56.229Z","trace_propagation_headers":{"sentry-trace":"b55a5ed2b2c345bda3ae125e4ee005f2-c64bd23be38949dc","baggage":"sentry-trace_id=b55a5ed2b2c345bda3ae125e4ee005f2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5JG9J37MPYF1JVFTYE7GE","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:38:56.231Z","job_size_bytes":185,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-f3a3310beeacf9891f6c3d7b: start","job_status":"start","queue_duration_s":3.340131,"scheduling_latency_s":3.340163} gitlab # [ 252.276329] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:59.581Z","message":"changes: \"2926555362c994dd725f8b6ab88adcd6b3aca0df d2fb939f2bad435ffeda072a310aa498ab167846 refs/heads/master\\n\""} gitlab # [ 252.302382] gitaly[918]: time="2026-09-13T10:38:59.607Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:39:09.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:59.594" grpc.time_ms=12.151 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.317741] gitaly[918]: time="2026-09-13T10:38:59.622Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:39:09.024" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:59.614" grpc.time_ms=7.633 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.333869] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:59.638Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"b5d64e3a4d22a577472d4047","created_at":"2026-09-13T10:38:59.636Z","trace_propagation_headers":{"sentry-trace":"b55a5ed2b2c345bda3ae125e4ee005f2-2b70d1b0a7024503","baggage":"sentry-trace_id=b55a5ed2b2c345bda3ae125e4ee005f2,sentry-release=Unknown"},"correlation_id":"01M2D5JG9J37MPYF1JVFTYE7GE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"75046fbc76050b4e047d6329","job_size_bytes":46,"pid":2556,"job_status":"deduplicated","message":"ProjectCacheWorker JID-b5d64e3a4d22a577472d4047: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 252.351623] gitaly[918]: time="2026-09-13T10:38:59.656Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-13T10:39:09.058" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:59.648" grpc.time_ms=7.581 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.366091] gitaly[918]: time="2026-09-13T10:38:59.670Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-13T10:39:29.560" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:59.660" grpc.time_ms=9.577 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.385996] gitaly[918]: time="2026-09-13T10:38:59.690Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=345 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-13T10:39:09.094" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-13T10:38:59.685" grpc.time_ms=5.222 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.406056] gitaly[918]: time="2026-09-13T10:38:59.710Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:39:09.113" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:38:59.703" grpc.time_ms=7.326 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.523186] gitaly[918]: time="2026-09-13T10:38:59.827Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-13T10:39:09.235" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:59.826" grpc.time_ms=1.302 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.544810] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:59.847Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","2926555362c994dd725f8b6ab88adcd6b3aca0df d2fb939f2bad435ffeda072a310aa498ab167846 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"f3a3310beeacf9891f6c3d7b","created_at":"2026-09-13T10:38:56.229Z","trace_propagation_headers":{"sentry-trace":"b55a5ed2b2c345bda3ae125e4ee005f2-c64bd23be38949dc","baggage":"sentry-trace_id=b55a5ed2b2c345bda3ae125e4ee005f2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5JG9J37MPYF1JVFTYE7GE","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:38:56.231Z","job_size_bytes":185,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-f3a3310beeacf9891f6c3d7b: done: 0.276556 sec","job_status":"done","queue_duration_s":3.340131,"scheduling_latency_s":3.340163,"gitaly_calls":7,"gitaly_duration_s":0.093048,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.024626000000000002,"redis_read_bytes":501,"redis_write_bytes":11943,"redis_cache_calls":9,"redis_cache_duration_s":0.00302,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001126,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5720,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005674,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.00607,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.008736,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.03,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.03,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.143186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000448661,"exclusive_lock_hold_duration_s":0.020780294,"duration_s":0.276556,"completed_at":"2026-09-13T10:38:59.847Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027523,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 252.592439] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:59.851Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"75046fbc76050b4e047d6329","created_at":"2026-09-13T10:38:57.825Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-6711849574e545a7","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:57.841Z","job_size_bytes":46,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-75046fbc76050b4e047d6329: start","job_status":"start","queue_duration_s":2.009794,"scheduling_latency_s":2.009837} gitlab # [ 252.606702] gitaly[918]: time="2026-09-13T10:38:59.911Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-13T16:38:59.905" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:38:59.906" grpc.time_ms=4.667 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.623874] gitaly[918]: time="2026-09-13T10:38:59.928Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-13T10:39:29.817" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:59.918" grpc.time_ms=10.012 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.645927] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:59.950Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"75046fbc76050b4e047d6329","created_at":"2026-09-13T10:38:57.825Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-6711849574e545a7","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:57.841Z","job_size_bytes":46,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-75046fbc76050b4e047d6329: done: 0.099325 sec","job_status":"done","queue_duration_s":2.009794,"scheduling_latency_s":2.009837,"gitaly_calls":2,"gitaly_duration_s":0.02033,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007716,"redis_read_bytes":69,"redis_write_bytes":4878,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001531,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000559,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2111,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001767,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.00303,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":748,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000829,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000275733,"duration_s":0.099325,"completed_at":"2026-09-13T10:38:59.950Z","load_balancing_strategy":"primary","db_duration_s":0.012769,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 252.650137] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:38:59.951Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"311c426194188bb1c3905ccc","created_at":"2026-09-13T10:38:57.853Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-6711849574e545a7","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:57.862Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-311c426194188bb1c3905ccc: start","job_status":"start","queue_duration_s":2.088792,"scheduling_latency_s":2.088807} gitlab # [ 252.697411] gitaly[918]: time="2026-09-13T10:39:00.002Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF diskcache=0953b239-6839-4dc3-b6c9-466ad4b2090a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T16:38:59.969" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:59.969" pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.698430] gitaly[918]: time="2026-09-13T10:39:00.002Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1279 command.oublock=0 command.real_time_ms=34 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T16:38:59.969" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:38:59.969" grpc.time_ms=32.508 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.729296] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:00.033Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"311c426194188bb1c3905ccc","created_at":"2026-09-13T10:38:57.853Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-6711849574e545a7","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:57.862Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-311c426194188bb1c3905ccc: done: 0.082143 sec","job_status":"done","queue_duration_s":2.088792,"scheduling_latency_s":2.088807,"gitaly_calls":1,"gitaly_duration_s":0.034844,"redis_calls":15,"redis_duration_s":0.003611,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000444,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001379,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000655,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001133,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026461,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.06458306800000001,"duration_s":0.082143,"completed_at":"2026-09-13T10:39:00.033Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016101,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 252.731913] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:00.035Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"0c6e95f57306c6b9837d9f2f","created_at":"2026-09-13T10:38:57.998Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-6711849574e545a7","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:58.000Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0c6e95f57306c6b9837d9f2f: start","job_status":"start","queue_duration_s":2.034479,"scheduling_latency_s":2.034494} gitlab # [ 252.798240] gitaly[918]: time="2026-09-13T10:39:00.102Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-13T10:39:30.090" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:00.091" grpc.time_ms=10.771 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.815447] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:00.119Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"0c6e95f57306c6b9837d9f2f","created_at":"2026-09-13T10:38:57.998Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-6711849574e545a7","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:58.000Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0c6e95f57306c6b9837d9f2f: done: 0.084189 sec","job_status":"done","queue_duration_s":2.034479,"scheduling_latency_s":2.034494,"gitaly_calls":1,"gitaly_duration_s":0.049415,"redis_calls":16,"redis_duration_s":0.004268,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000439,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00131,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001777,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000742,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025237,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.084189,"completed_at":"2026-09-13T10:39:00.119Z","load_balancing_strategy":"primary","db_duration_s":0.002876,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 252.818115] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:00.122Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2140aa33824bdbcd698a5065","created_at":"2026-09-13T10:38:58.493Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-6711849574e545a7","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:41f4b933e2f14c83896663e8b000e57cde134a714d81c1139ff4e78499eda83b","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:58.498Z","job_size_bytes":467,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2140aa33824bdbcd698a5065: start","job_status":"start","queue_duration_s":1.623891,"scheduling_latency_s":1.62393} gitlab # [ 252.867711] gitaly[918]: time="2026-09-13T10:39:00.161Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:30.059" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:00.159" grpc.time_ms=1.092 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.870548] gitaly[918]: time="2026-09-13T10:39:00.175Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T10:39:10.002" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:00.163" grpc.time_ms=11.442 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.989511] gitaly[918]: time="2026-09-13T10:39:00.293Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:39:30.291" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:00.292" grpc.time_ms=1.315 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.016383] gitaly[918]: time="2026-09-13T10:39:00.321Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=11 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-13T10:39:30.316" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:00.316" grpc.time_ms=4.056 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.111633] gitaly[918]: time="2026-09-13T10:39:00.416Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:39:10.244" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:00.404" grpc.time_ms=11.506 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846' gitlab # [ 253.291488] nixbot[2669]: INFO: ::ffff:127.0.0.1:33718 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846', in 0.05 seconds) gitlab # [ 253.616152] gitaly[918]: time="2026-09-13T10:39:00.920Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-13T10:39:30.004" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:00.904" grpc.time_ms=15.967 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.636740] gitaly[918]: time="2026-09-13T10:39:00.941Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2D5H42KY41M09SFZRR7F1XF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-13T10:39:30.025" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:00.925" grpc.time_ms=15.945 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 253.689311] sidekiq-cluster[2556]: {"severity":"WARN","time":"2026-09-13T10:39:00.993Z","class":"CreatePipelineWorker","correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"2926555362c994dd725f8b6ab88adcd6b3aca0df","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"2926555362c994dd725f8b6ab88adcd6b3aca0df","ref":"refs/heads/master"},"retry":0} gitlab # [ 253.694711] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:00.998Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2140aa33824bdbcd698a5065","created_at":"2026-09-13T10:38:58.493Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-6711849574e545a7","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:41f4b933e2f14c83896663e8b000e57cde134a714d81c1139ff4e78499eda83b","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:58.498Z","job_size_bytes":467,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2140aa33824bdbcd698a5065: done: 0.876263 sec","job_status":"done","queue_duration_s":1.623891,"scheduling_latency_s":1.62393,"gitaly_calls":7,"gitaly_duration_s":0.149982,"redis_calls":51,"redis_duration_s":0.012286,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.001345,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004956,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001714,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003163,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001108,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.66979,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002992,"duration_s":0.876263,"completed_at":"2026-09-13T10:39:00.998Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.056453,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 253.754066] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.000Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"1eb7fb7b-5389-4b64-996c-03df24b933b8\"}"],"class":"WebHookWorker","jid":"b3e9de44890285cade9d63a9","created_at":"2026-09-13T10:38:58.597Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-6711849574e545a7","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2945660bd7249fc82fd277712ae203e03d5d844f8f6e4ceaa639fdd963cf0878","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:58.602Z","job_size_bytes":1751,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-b3e9de44890285cade9d63a9: start","job_status":"start","queue_duration_s":2.398307,"scheduling_latency_s":2.398321} gitlab # [ 253.819852] nixbot[2669]: INFO: ::ffff:127.0.0.1:33730 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 253.847807] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.151Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"1eb7fb7b-5389-4b64-996c-03df24b933b8\"}"],"class":"WebHookWorker","jid":"b3e9de44890285cade9d63a9","created_at":"2026-09-13T10:38:58.597Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-6711849574e545a7","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2945660bd7249fc82fd277712ae203e03d5d844f8f6e4ceaa639fdd963cf0878","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:58.602Z","job_size_bytes":1751,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-b3e9de44890285cade9d63a9: done: 0.151087 sec","job_status":"done","queue_duration_s":2.398307,"scheduling_latency_s":2.398321,"redis_calls":19,"redis_duration_s":0.0044,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001085,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000543,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001909,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000863,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.058555766000012,"cpu_s":0.077548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000544203,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.151087,"completed_at":"2026-09-13T10:39:01.151Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002927,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.858241] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.154Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"ef446f21413c9f6edde98c52","created_at":"2026-09-13T10:38:59.023Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.027Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-ef446f21413c9f6edde98c52: start","job_status":"start","queue_duration_s":2.126389,"scheduling_latency_s":2.126432} gitlab # [ 253.920478] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.224Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"ef446f21413c9f6edde98c52","created_at":"2026-09-13T10:38:59.023Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.027Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-ef446f21413c9f6edde98c52: done: 0.070883 sec","job_status":"done","queue_duration_s":2.126389,"scheduling_latency_s":2.126432,"redis_calls":10,"redis_duration_s":0.001838,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000366,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0011,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000372,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000224331,"duration_s":0.070883,"completed_at":"2026-09-13T10:39:01.224Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011649,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 253.922367] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.225Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"02c2b7850147c3f7bb6597ba","created_at":"2026-09-13T10:38:59.224Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.231Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-02c2b7850147c3f7bb6597ba: start","job_status":"start","queue_duration_s":1.993919,"scheduling_latency_s":1.993934} gitlab # [ 253.943876] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.248Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"02c2b7850147c3f7bb6597ba","created_at":"2026-09-13T10:38:59.224Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.231Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-02c2b7850147c3f7bb6597ba: done: 0.022496 sec","job_status":"done","queue_duration_s":1.993919,"scheduling_latency_s":1.993934,"redis_calls":10,"redis_duration_s":0.001418,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000283,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000841,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000294,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000235226,"duration_s":0.022496,"completed_at":"2026-09-13T10:39:01.248Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.986495] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.249Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"4d743333184436ff9efe0525","created_at":"2026-09-13T10:38:59.248Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.255Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-4d743333184436ff9efe0525: start","job_status":"start","queue_duration_s":1.993733,"scheduling_latency_s":1.993747} gitlab # [ 254.025646] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.271Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"4d743333184436ff9efe0525","created_at":"2026-09-13T10:38:59.248Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.255Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-4d743333184436ff9efe0525: done: 0.02214 sec","job_status":"done","queue_duration_s":1.993733,"scheduling_latency_s":1.993747,"redis_calls":10,"redis_duration_s":0.001418,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000277,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000216787,"duration_s":0.02214,"completed_at":"2026-09-13T10:39:01.271Z","load_balancing_strategy":"primary","db_duration_s":0.003179,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.066225] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.273Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4f707af20279f2c74b60cfca","created_at":"2026-09-13T10:38:59.277Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.284Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4f707af20279f2c74b60cfca: start","job_status":"start","queue_duration_s":1.987999,"scheduling_latency_s":1.988013} gitlab # [ 254.067211] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.297Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4f707af20279f2c74b60cfca","created_at":"2026-09-13T10:38:59.277Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.284Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4f707af20279f2c74b60cfca: done: 0.024273 sec","job_status":"done","queue_duration_s":1.987999,"scheduling_latency_s":1.988013,"redis_calls":13,"redis_duration_s":0.001756,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000301,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000794,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000392,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000269,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022014,"duration_s":0.024273,"completed_at":"2026-09-13T10:39:01.297Z","load_balancing_strategy":"primary","db_duration_s":0.003247,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.109996] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.298Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"2e68c49220d8dea2e12f9cf6","created_at":"2026-09-13T10:38:59.285Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.293Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-2e68c49220d8dea2e12f9cf6: start","job_status":"start","queue_duration_s":2.005047,"scheduling_latency_s":2.005062} gitlab # [ 254.110775] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.319Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"2e68c49220d8dea2e12f9cf6","created_at":"2026-09-13T10:38:59.285Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.293Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-2e68c49220d8dea2e12f9cf6: done: 0.021523 sec","job_status":"done","queue_duration_s":2.005047,"scheduling_latency_s":2.005062,"redis_calls":10,"redis_duration_s":0.001375,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000308,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000806,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000213435,"duration_s":0.021523,"completed_at":"2026-09-13T10:39:01.319Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002747,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.149193] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.320Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"5ff88edfa1a0accc44a45d62","created_at":"2026-09-13T10:38:59.294Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.300Z","job_size_bytes":19,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-5ff88edfa1a0accc44a45d62: start","job_status":"start","queue_duration_s":2.020503,"scheduling_latency_s":2.020517} gitlab # [ 254.151617] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.341Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"5ff88edfa1a0accc44a45d62","created_at":"2026-09-13T10:38:59.294Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.300Z","job_size_bytes":19,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-5ff88edfa1a0accc44a45d62: done: 0.020576 sec","job_status":"done","queue_duration_s":2.020503,"scheduling_latency_s":2.020517,"redis_calls":11,"redis_duration_s":0.001522,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00099,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000227962,"duration_s":0.020576,"completed_at":"2026-09-13T10:39:01.341Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002538,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.194796] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.342Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ae65a35dd967a2131f7258e3","created_at":"2026-09-13T10:38:59.304Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.309Z","job_size_bytes":66,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ae65a35dd967a2131f7258e3: start","job_status":"start","queue_duration_s":2.033494,"scheduling_latency_s":2.033509} gitlab # [ 254.206761] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.439Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ae65a35dd967a2131f7258e3","created_at":"2026-09-13T10:38:59.304Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.309Z","job_size_bytes":66,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ae65a35dd967a2131f7258e3: done: 0.096572 sec","job_status":"done","queue_duration_s":2.033494,"scheduling_latency_s":2.033509,"redis_calls":10,"redis_duration_s":0.003172,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000872,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001844,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.068,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.064,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020582,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000369041,"duration_s":0.096572,"completed_at":"2026-09-13T10:39:01.439Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.069854,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846' gitlab # [ 254.275633] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.440Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"da08149476b32c473637f84d","created_at":"2026-09-13T10:38:59.310Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.314Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-da08149476b32c473637f84d: start","job_status":"start","queue_duration_s":2.125829,"scheduling_latency_s":2.125844} gitlab # [ 254.276666] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.493Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"da08149476b32c473637f84d","created_at":"2026-09-13T10:38:59.310Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.314Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-da08149476b32c473637f84d: done: 0.052932 sec","job_status":"done","queue_duration_s":2.125829,"scheduling_latency_s":2.125844,"redis_calls":10,"redis_duration_s":0.0024490000000000002,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000524,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001459,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000466,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000412343,"duration_s":0.052932,"completed_at":"2026-09-13T10:39:01.493Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007927,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.318711] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.495Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"3dc6e4f1fd3dc76d9f01bd7d","created_at":"2026-09-13T10:38:59.321Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.326Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-3dc6e4f1fd3dc76d9f01bd7d: start","job_status":"start","queue_duration_s":2.169357,"scheduling_latency_s":2.169386} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846', in 0.07 seconds) gitlab # [ 254.319671] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.547Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"3dc6e4f1fd3dc76d9f01bd7d","created_at":"2026-09-13T10:38:59.321Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.326Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-3dc6e4f1fd3dc76d9f01bd7d: done: 0.052418 sec","job_status":"done","queue_duration_s":2.169357,"scheduling_latency_s":2.169386,"redis_calls":10,"redis_duration_s":0.006397,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000767,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004505,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001125,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000719645,"duration_s":0.052418,"completed_at":"2026-09-13T10:39:01.547Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016103,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.366682] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.549Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"7d30102f44f39b1a73eb102b","created_at":"2026-09-13T10:38:59.326Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.330Z","job_size_bytes":6,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-7d30102f44f39b1a73eb102b: start","job_status":"start","queue_duration_s":2.219464,"scheduling_latency_s":2.219479} gitlab # [ 254.370283] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.567Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"7d30102f44f39b1a73eb102b","created_at":"2026-09-13T10:38:59.326Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.330Z","job_size_bytes":6,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-7d30102f44f39b1a73eb102b: done: 0.018099 sec","job_status":"done","queue_duration_s":2.219464,"scheduling_latency_s":2.219479,"redis_calls":10,"redis_duration_s":0.0021579999999999998,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000404,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0013,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000454,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000350882,"duration_s":0.018099,"completed_at":"2026-09-13T10:39:01.567Z","load_balancing_strategy":"primary","db_duration_s":0.001993,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.445817] nixbot[2669]: INFO: ::ffff:127.0.0.1:33738 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846 HTTP/1.1" 200 OK gitlab # [ 254.446937] gitaly[918]: time="2026-09-13T10:39:01.701Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE diskcache=dfd82986-f127-44b2-8639-1ac3bab423d8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T16:39:01.691" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:01.692" pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.447971] gitaly[918]: time="2026-09-13T10:39:01.701Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=317 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T16:39:01.691" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:01.692" grpc.time_ms=9.588 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.448628] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.569Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"c5bd3388dfd82fa75fe9bb91","created_at":"2026-09-13T10:38:59.330Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.333Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-c5bd3388dfd82fa75fe9bb91: start","job_status":"start","queue_duration_s":2.235806,"scheduling_latency_s":2.235819} gitlab # [ 254.449207] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.593Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"c5bd3388dfd82fa75fe9bb91","created_at":"2026-09-13T10:38:59.330Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.333Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-c5bd3388dfd82fa75fe9bb91: done: 0.024527 sec","job_status":"done","queue_duration_s":2.235806,"scheduling_latency_s":2.235819,"redis_calls":10,"redis_duration_s":0.002194,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000431,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001304,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000459,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01478,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000289422,"duration_s":0.024527,"completed_at":"2026-09-13T10:39:01.593Z","load_balancing_strategy":"primary","db_duration_s":0.004557,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.491926] gitaly[918]: time="2026-09-13T10:39:01.731Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:31.730" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:01.730" grpc.time_ms=0.583 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.497082] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.594Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"0edc3b6bd33a19bf594ba8ba","created_at":"2026-09-13T10:38:59.338Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.418Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-0edc3b6bd33a19bf594ba8ba: start","job_status":"start","queue_duration_s":2.176398,"scheduling_latency_s":2.176411} gitlab # [ 254.545634] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.644Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"0edc3b6bd33a19bf594ba8ba","created_at":"2026-09-13T10:38:59.338Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.418Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-0edc3b6bd33a19bf594ba8ba: done: 0.049824 sec","job_status":"done","queue_duration_s":2.176398,"scheduling_latency_s":2.176411,"redis_calls":14,"redis_duration_s":0.0065769999999999995,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000494,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001601,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004482,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000324064,"duration_s":0.049824,"completed_at":"2026-09-13T10:39:01.644Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009205,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.549187] gitaly[918]: time="2026-09-13T10:39:01.741Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T10:39:11.144" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:01.734" grpc.time_ms=6.795 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.586900] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.646Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"4b69cd446c6521d576ab7c34","created_at":"2026-09-13T10:38:59.491Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.513Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4b69cd446c6521d576ab7c34: start","job_status":"start","queue_duration_s":2.13235,"scheduling_latency_s":2.132363} gitlab # [ 254.587708] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.667Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"4b69cd446c6521d576ab7c34","created_at":"2026-09-13T10:38:59.491Z","trace_propagation_headers":{"sentry-trace":"edc34c605fe84f70b0778ab42e8e38f0-2a6af3cf74cb4354","baggage":"sentry-trace_id=edc34c605fe84f70b0778ab42e8e38f0,sentry-release=Unknown"},"correlation_id":"01M2D5H955G51KR0KEH017P1WA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.513Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4b69cd446c6521d576ab7c34: done: 0.021311 sec","job_status":"done","queue_duration_s":2.13235,"scheduling_latency_s":2.132363,"redis_calls":8,"redis_duration_s":0.002104,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001426,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000678,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021311,"completed_at":"2026-09-13T10:39:01.667Z","load_balancing_strategy":"primary","db_duration_s":0.005362,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 254.634663] gitaly[918]: time="2026-09-13T10:39:01.768Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:39:31.766" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:01.767" grpc.time_ms=0.769 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.635254] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.668Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"782433f0883bd52335b74791","created_at":"2026-09-13T10:38:59.643Z","trace_propagation_headers":{"sentry-trace":"b55a5ed2b2c345bda3ae125e4ee005f2-2b70d1b0a7024503","baggage":"sentry-trace_id=b55a5ed2b2c345bda3ae125e4ee005f2,sentry-release=Unknown"},"correlation_id":"01M2D5JG9J37MPYF1JVFTYE7GE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"311c426194188bb1c3905ccc","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.645Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-782433f0883bd52335b74791: start","job_status":"start","queue_duration_s":2.023254,"scheduling_latency_s":2.023269} gitlab # [ 254.636734] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.715Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"782433f0883bd52335b74791","created_at":"2026-09-13T10:38:59.643Z","trace_propagation_headers":{"sentry-trace":"b55a5ed2b2c345bda3ae125e4ee005f2-2b70d1b0a7024503","baggage":"sentry-trace_id=b55a5ed2b2c345bda3ae125e4ee005f2,sentry-release=Unknown"},"correlation_id":"01M2D5JG9J37MPYF1JVFTYE7GE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"311c426194188bb1c3905ccc","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.645Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-782433f0883bd52335b74791: done: 0.046466 sec","job_status":"done","queue_duration_s":2.023254,"scheduling_latency_s":2.023269,"gitaly_calls":1,"gitaly_duration_s":0.011784,"redis_calls":12,"redis_duration_s":0.008517,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002343,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00262,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003554,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.026979127,"duration_s":0.046466,"completed_at":"2026-09-13T10:39:01.715Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010086,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.686251] gitaly[918]: time="2026-09-13T10:39:01.795Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=39 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-13T10:39:31.772" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:01.772" grpc.time_ms=23.347 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.691979] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:01.716Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2b1e4db0985cb2299c94d401","created_at":"2026-09-13T10:38:59.779Z","trace_propagation_headers":{"sentry-trace":"b55a5ed2b2c345bda3ae125e4ee005f2-2b70d1b0a7024503","baggage":"sentry-trace_id=b55a5ed2b2c345bda3ae125e4ee005f2,sentry-release=Unknown"},"correlation_id":"01M2D5JG9J37MPYF1JVFTYE7GE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1ad0345644586a4189427e0f0e592d11a29d658b4b788a45867bcece0f81d0a3","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.783Z","job_size_bytes":467,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2b1e4db0985cb2299c94d401: start","job_status":"start","queue_duration_s":1.932894,"scheduling_latency_s":1.932907} gitlab # [ 254.736484] gitaly[918]: time="2026-09-13T10:39:01.846Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:39:11.239" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:01.830" grpc.time_ms=16.105 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.801127] gitaly[918]: time="2026-09-13T10:39:02.104Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JNGV4QG4SV2X37P2R4TQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:02.097" grpc.time_ms=7.073 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 254.802094] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:02 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 958 gitlab # [ 254.865761] gitaly[918]: time="2026-09-13T10:39:02.170Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPEVR84F3CY3VWST7Q16 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:02.163" pid=918 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 254.866848] gitaly[918]: time="2026-09-13T10:39:02.170Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPEVR84F3CY3VWST7Q16 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:02.163" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 254.867642] gitlab-workhorse[829]: time="2026-09-13T10:39:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 254.868537] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:02 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 63 gitlab # [ 254.869734] gitaly[918]: time="2026-09-13T10:39:02.171Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPEVR84F3CY3VWST7Q16 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:02.163" grpc.time_ms=7.348 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 254.919520] gitaly[918]: time="2026-09-13T10:39:02.224Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPGXZ4V7C6TWPTTFDKES grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:02.211" pid=918 remote_ip="2001:db8:1::1" request_sha=c07dbf32423c18101bb4dd90d8d36369b6707b1b response_bytes=337 span.kind=server system=grpc gitlab # [ 254.920292] gitaly[918]: time="2026-09-13T10:39:02.224Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1190 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPGXZ4V7C6TWPTTFDKES grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:02.211" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 254.941445] gitaly[918]: time="2026-09-13T10:39:02.246Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1190 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPGXZ4V7C6TWPTTFDKES grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:02.211" grpc.time_ms=34.41 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 254.953448] gitlab-workhorse[829]: time="2026-09-13T10:39:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=337 gitlab # [ 254.954158] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:02 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 84 gitlab # [ 254.964420] gitaly[918]: time="2026-09-13T10:39:02.269Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=13 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=13 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=958 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-13T10:39:32.245" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:02.245" grpc.time_ms=23.069 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.976262] gitaly[918]: time="2026-09-13T10:39:02.280Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JG9J37MPYF1JVFTYE7GE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-13T10:39:32.273" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:02.273" grpc.time_ms=7.265 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.987808] sidekiq-cluster[2556]: {"severity":"WARN","time":"2026-09-13T10:39:02.292Z","class":"CreatePipelineWorker","correlation_id":"01M2D5JG9J37MPYF1JVFTYE7GE","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"2926555362c994dd725f8b6ab88adcd6b3aca0df","after":"d2fb939f2bad435ffeda072a310aa498ab167846","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"d2fb939f2bad435ffeda072a310aa498ab167846","ref":"refs/heads/master"},"retry":0} gitlab # [ 254.992922] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:02.297Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2b1e4db0985cb2299c94d401","created_at":"2026-09-13T10:38:59.779Z","trace_propagation_headers":{"sentry-trace":"b55a5ed2b2c345bda3ae125e4ee005f2-2b70d1b0a7024503","baggage":"sentry-trace_id=b55a5ed2b2c345bda3ae125e4ee005f2,sentry-release=Unknown"},"correlation_id":"01M2D5JG9J37MPYF1JVFTYE7GE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1ad0345644586a4189427e0f0e592d11a29d658b4b788a45867bcece0f81d0a3","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.783Z","job_size_bytes":467,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2b1e4db0985cb2299c94d401: done: 0.581208 sec","job_status":"done","queue_duration_s":1.932894,"scheduling_latency_s":1.932907,"gitaly_calls":7,"gitaly_duration_s":0.097147,"redis_calls":33,"redis_duration_s":0.006759,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.00329,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001819,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000771,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.458328,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.581208,"completed_at":"2026-09-13T10:39:02.297Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014905,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 254.999207] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:02.298Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"a3f0db0c-7842-4061-9eff-a649515e64c3\"}"],"class":"WebHookWorker","jid":"d564237508c01401e4134d3f","created_at":"2026-09-13T10:38:59.806Z","trace_propagation_headers":{"sentry-trace":"b55a5ed2b2c345bda3ae125e4ee005f2-2b70d1b0a7024503","baggage":"sentry-trace_id=b55a5ed2b2c345bda3ae125e4ee005f2,sentry-release=Unknown"},"correlation_id":"01M2D5JG9J37MPYF1JVFTYE7GE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0f4b572b99db95ea86a65dbaab4ae95f7101119da8991727e4b4082d31cc9ef3","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.810Z","job_size_bytes":1749,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-d564237508c01401e4134d3f: start","job_status":"start","queue_duration_s":2.487997,"scheduling_latency_s":2.488017} gitlab # [ 255.019667] nixbot[2669]: INFO: ::ffff:127.0.0.1:33750 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 255.027421] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:02.331Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"a3f0db0c-7842-4061-9eff-a649515e64c3\"}"],"class":"WebHookWorker","jid":"d564237508c01401e4134d3f","created_at":"2026-09-13T10:38:59.806Z","trace_propagation_headers":{"sentry-trace":"b55a5ed2b2c345bda3ae125e4ee005f2-2b70d1b0a7024503","baggage":"sentry-trace_id=b55a5ed2b2c345bda3ae125e4ee005f2,sentry-release=Unknown"},"correlation_id":"01M2D5JG9J37MPYF1JVFTYE7GE","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0f4b572b99db95ea86a65dbaab4ae95f7101119da8991727e4b4082d31cc9ef3","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.810Z","job_size_bytes":1749,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-d564237508c01401e4134d3f: done: 0.033059 sec","job_status":"done","queue_duration_s":2.487997,"scheduling_latency_s":2.488017,"redis_calls":14,"redis_duration_s":0.00249,"redis_read_bytes":25,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000221,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001534,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000735,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.0058158230000060485,"cpu_s":0.020615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.033059,"completed_at":"2026-09-13T10:39:02.331Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002766,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.086100] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:02.333Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"6f5db9b2bb3beb6573114360","created_at":"2026-09-13T10:38:59.937Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-9dae1775910c463b","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.941Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-6f5db9b2bb3beb6573114360: start","job_status":"start","queue_duration_s":2.391491,"scheduling_latency_s":2.391522} gitlab # [ 255.106083] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:02.410Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"6f5db9b2bb3beb6573114360","created_at":"2026-09-13T10:38:59.937Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-9dae1775910c463b","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-13T10:38:59.941Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-6f5db9b2bb3beb6573114360: done: 0.077373 sec","job_status":"done","queue_duration_s":2.391491,"scheduling_latency_s":2.391522,"redis_calls":19,"redis_duration_s":0.04228,"redis_read_bytes":21,"redis_write_bytes":4288,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000597,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000381,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2259,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001407,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.039895,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00886202,"duration_s":0.077373,"completed_at":"2026-09-13T10:39:02.410Z","load_balancing_strategy":"primary","db_duration_s":0.00496,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.113697] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:02.412Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","b3e9de44890285cade9d63a9"],"class":"WebHooks::LogExecutionWorker","jid":"a1e7343447adf1ec7f38bc37","created_at":"2026-09-13T10:39:01.132Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-29c02726c69641a2","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b57b26a35c751c89c8a354a3c75e06fc2dc3bd08f3bc90e9234a707ae108511","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:01.140Z","job_size_bytes":2593,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-a1e7343447adf1ec7f38bc37: start","job_status":"start","queue_duration_s":1.271967,"scheduling_latency_s":1.271996} gitlab # [ 255.148156] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:02.452Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","b3e9de44890285cade9d63a9"],"class":"WebHooks::LogExecutionWorker","jid":"a1e7343447adf1ec7f38bc37","created_at":"2026-09-13T10:39:01.132Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-29c02726c69641a2","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b57b26a35c751c89c8a354a3c75e06fc2dc3bd08f3bc90e9234a707ae108511","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:01.140Z","job_size_bytes":2593,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-a1e7343447adf1ec7f38bc37: done: 0.040462 sec","job_status":"done","queue_duration_s":1.271967,"scheduling_latency_s":1.271996,"redis_calls":10,"redis_duration_s":0.001648,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000368,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000986,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000294,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025115,"duration_s":0.040462,"completed_at":"2026-09-13T10:39:02.452Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00993,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.197035] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:02.453Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","d564237508c01401e4134d3f"],"class":"WebHooks::LogExecutionWorker","jid":"8ad0247db82bfe1ccb5003e7","created_at":"2026-09-13T10:39:02.325Z","trace_propagation_headers":{"sentry-trace":"b55a5ed2b2c345bda3ae125e4ee005f2-741b00473edb4a3a","baggage":"sentry-trace_id=b55a5ed2b2c345bda3ae125e4ee005f2,sentry-release=Unknown"},"correlation_id":"01M2D5JG9J37MPYF1JVFTYE7GE","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a5f8702953fda50abd076ab3a15a0f32d5955f57969ecb3088e718d7fe1d4975","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.327Z","job_size_bytes":2592,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-8ad0247db82bfe1ccb5003e7: start","job_status":"start","queue_duration_s":0.126192,"scheduling_latency_s":0.126206} gitlab # [ 255.197857] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:02.471Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","d564237508c01401e4134d3f"],"class":"WebHooks::LogExecutionWorker","jid":"8ad0247db82bfe1ccb5003e7","created_at":"2026-09-13T10:39:02.325Z","trace_propagation_headers":{"sentry-trace":"b55a5ed2b2c345bda3ae125e4ee005f2-741b00473edb4a3a","baggage":"sentry-trace_id=b55a5ed2b2c345bda3ae125e4ee005f2,sentry-release=Unknown"},"correlation_id":"01M2D5JG9J37MPYF1JVFTYE7GE","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a5f8702953fda50abd076ab3a15a0f32d5955f57969ecb3088e718d7fe1d4975","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.327Z","job_size_bytes":2592,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-8ad0247db82bfe1ccb5003e7: done: 0.018183 sec","job_status":"done","queue_duration_s":0.126192,"scheduling_latency_s":0.126206,"redis_calls":7,"redis_duration_s":0.001046,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000758,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000288,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018183,"completed_at":"2026-09-13T10:39:02.471Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006437,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.249959] nixbot[2669]: {"ts": "2026-09-13T10:39:02.457Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 255.250734] gitaly[918]: time="2026-09-13T10:39:02.514Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:32.513" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:02.513" grpc.time_ms=0.878 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.251583] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:02.472Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"4a53271e27a3c7a5e2760567","created_at":"2026-09-13T10:39:02.396Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-7bef83a737594677","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.402Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-4a53271e27a3c7a5e2760567: start","job_status":"start","queue_duration_s":0.069913,"scheduling_latency_s":0.069927} gitlab # [ 255.253120] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:02.532Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"4a53271e27a3c7a5e2760567","created_at":"2026-09-13T10:39:02.396Z","trace_propagation_headers":{"sentry-trace":"cc9e230b08bc44fe871d47c5078f8e27-7bef83a737594677","baggage":"sentry-trace_id=cc9e230b08bc44fe871d47c5078f8e27,sentry-release=Unknown"},"correlation_id":"01M2D5H42KY41M09SFZRR7F1XF","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.402Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-4a53271e27a3c7a5e2760567: done: 0.059782 sec","job_status":"done","queue_duration_s":0.069913,"scheduling_latency_s":0.069927,"redis_calls":11,"redis_duration_s":0.00171,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001122,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000316,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000238299,"duration_s":0.059782,"completed_at":"2026-09-13T10:39:02.532Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017187,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.301914] gitaly[918]: time="2026-09-13T10:39:02.584Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:39:12.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:02.577" grpc.time_ms=6.846 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.313574] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:02.618Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"aafbfc5c6957673a31b6fe40","created_at":"2026-09-13T10:39:02.610Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-60715f244f39494e","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.616Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-aafbfc5c6957673a31b6fe40: start","job_status":"start","queue_duration_s":0.001847,"scheduling_latency_s":0.001876} gitlab # [ 255.354878] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2926555362c994dd725f8b6ab88adcd6b3aca0df HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 161 gitlab # [ 255.355577] nixbot[2669]: {"ts": "2026-09-13T10:39:02.632Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2926555362c994dd725f8b6ab88adcd6b3aca0df \"HTTP/1.1 201 Created\""} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846' gitlab # [ 255.370251] gitaly[918]: time="2026-09-13T10:39:02.674Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPZBKAFBEY2EC6E0DV6N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:32.173" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:02.674" grpc.time_ms=0.518 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.414461] nixbot[2669]: INFO: ::ffff:127.0.0.1:33764 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846', in 0.05 seconds) gitlab # [ 255.418797] gitaly[918]: time="2026-09-13T10:39:02.723Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JPZBKAFBEY2EC6E0DV6N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:39:12.143" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:02.714" grpc.time_ms=9.342 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.434386] gitaly[918]: time="2026-09-13T10:39:02.739Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=2 command.system_time_ms=16 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JQ0N9ZDDNM4TKD351Z0C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:02.718" grpc.time_ms=20.703 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 255.435810] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:02 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 61 gitlab # [ 255.437408] gitaly[918]: time="2026-09-13T10:39:02.742Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:39:02.741" grpc.time_ms=0.631 pid=918 span.kind=server system=grpc gitlab # [ 255.477434] gitaly[918]: time="2026-09-13T10:39:02.781Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:39:02.781" grpc.time_ms=0.441 pid=918 span.kind=server system=grpc gitlab # [ 255.483881] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2926555362c994dd725f8b6ab88adcd6b3aca0df HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 152 gitlab # [ 255.485034] nixbot[2669]: {"ts": "2026-09-13T10:39:02.788Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2926555362c994dd725f8b6ab88adcd6b3aca0df \"HTTP/1.1 201 Created\""} gitlab # [ 255.490826] gitaly[918]: time="2026-09-13T10:39:02.795Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQ2N4DSK86RMPRTQ0T81 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:02.787" pid=918 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 255.492148] gitaly[918]: time="2026-09-13T10:39:02.795Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQ2N4DSK86RMPRTQ0T81 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:02.787" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 255.492904] gitlab-workhorse[829]: time="2026-09-13T10:39:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 255.493532] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:02 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 54 gitlab # [ 255.494017] gitaly[918]: time="2026-09-13T10:39:02.795Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQ2N4DSK86RMPRTQ0T81 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:02.787" grpc.time_ms=8.211 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 255.555696] gitaly[918]: time="2026-09-13T10:39:02.860Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:39:02.859" grpc.time_ms=0.644 pid=918 span.kind=server system=grpc gitlab # [ 255.564514] gitaly[918]: time="2026-09-13T10:39:02.869Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQ494R9YPDJ7MX40EVZ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:32.868" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:02.868" grpc.time_ms=0.564 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.585926] gitaly[918]: time="2026-09-13T10:39:02.890Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 diskcache=f2752eac-3645-44cb-98bc-cc2be163742f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:39:32.712" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:02.712" pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.587111] gitaly[918]: time="2026-09-13T10:39:02.890Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=115 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7338 command.oublock=0 command.real_time_ms=169 command.spawn_token_fork_ms=1 command.system_time_ms=89 command.user_time_ms=26 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:39:32.712" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:02.712" grpc.time_ms=178.21 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.633858] gitaly[918]: time="2026-09-13T10:39:02.938Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:39:32.036" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:02.936" grpc.time_ms=0.826 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.640226] gitaly[918]: time="2026-09-13T10:39:02.944Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JQ5F6RAVGBWSJJC2DBBS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:02.936" grpc.time_ms=8.336 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 255.641639] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:02 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 115 gitlab # [ 255.647282] gitaly[918]: time="2026-09-13T10:39:02.952Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JQ494R9YPDJ7MX40EVZ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:39:12.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:02.944" grpc.time_ms=7.649 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.672870] gitaly[918]: time="2026-09-13T10:39:02.977Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:32.076" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:02.976" grpc.time_ms=0.597 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.685302] nixbot[2669]: {"ts": "2026-09-13T10:39:02.989Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/2926555362c994dd725f8b6ab88adcd6b3aca0df \"HTTP/1.1 201 Created\""} gitlab # [ 255.688185] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/2926555362c994dd725f8b6ab88adcd6b3aca0df HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 197 gitlab # [ 255.695236] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:02.999Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"d4210817a2551e46f2ef6ff3","created_at":"2026-09-13T10:39:02.997Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"aafbfc5c6957673a31b6fe40","job_size_bytes":3,"pid":2556,"job_status":"deduplicated","message":"PipelineProcessWorker JID-d4210817a2551e46f2ef6ff3: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 255.696386] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.001Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"aafbfc5c6957673a31b6fe40","created_at":"2026-09-13T10:39:02.610Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-60715f244f39494e","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.616Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2556,"job_status":"rescheduled","message":"PipelineProcessWorker JID-aafbfc5c6957673a31b6fe40: rescheduled"} gitlab # [ 255.698990] gitaly[918]: time="2026-09-13T10:39:03.003Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQ94W1R7FJNYNGNSS0X3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:02.989" pid=918 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 255.699730] gitaly[918]: time="2026-09-13T10:39:03.003Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQ94W1R7FJNYNGNSS0X3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:02.989" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 255.707843] gitaly[918]: time="2026-09-13T10:39:03.003Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1170 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQ94W1R7FJNYNGNSS0X3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:02.989" grpc.time_ms=14.173 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 255.753332] gitlab-workhorse[829]: time="2026-09-13T10:39:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 255.753961] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 63 gitlab # [ 255.754675] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.006Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"aafbfc5c6957673a31b6fe40","created_at":"2026-09-13T10:39:02.610Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-60715f244f39494e","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.616Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-aafbfc5c6957673a31b6fe40: done: 0.388188 sec","job_status":"done","queue_duration_s":0.001847,"scheduling_latency_s":0.001876,"gitaly_calls":3,"gitaly_duration_s":0.199227,"redis_calls":70,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.013341,"redis_read_bytes":227,"redis_write_bytes":18551,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000483,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001169,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.003316,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12312,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.005132,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000638,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000336,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001784,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.019,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.019,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.104988,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.346951638,"duration_s":0.388188,"completed_at":"2026-09-13T10:39:03.006Z","load_balancing_strategy":"primary","db_duration_s":0.046116,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 255.804098] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.008Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b38991e61a0a639fd5351a0d","created_at":"2026-09-13T10:39:02.703Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.707Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b38991e61a0a639fd5351a0d: start","job_status":"start","queue_duration_s":0.300886,"scheduling_latency_s":0.300904} gitlab # [ 255.804791] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.039Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b38991e61a0a639fd5351a0d","created_at":"2026-09-13T10:39:02.703Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.707Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b38991e61a0a639fd5351a0d: done: 0.031576 sec","job_status":"done","queue_duration_s":0.300886,"scheduling_latency_s":0.300904,"redis_calls":7,"redis_duration_s":0.0025009999999999998,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00224,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019906,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031576,"completed_at":"2026-09-13T10:39:03.039Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006435,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 255.859307] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.040Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7e4e30d7f21afe275902be6e","created_at":"2026-09-13T10:39:02.895Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.897Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7e4e30d7f21afe275902be6e: start","job_status":"start","queue_duration_s":0.143421,"scheduling_latency_s":0.143437} gitlab # [ 255.860105] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.057Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"7e4e30d7f21afe275902be6e","created_at":"2026-09-13T10:39:02.895Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.897Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-7e4e30d7f21afe275902be6e: done: 0.01689 sec","job_status":"done","queue_duration_s":0.143421,"scheduling_latency_s":0.143437,"redis_calls":7,"redis_duration_s":0.001024,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000772,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000252,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011332,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01689,"completed_at":"2026-09-13T10:39:03.057Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002639,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.919611] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.058Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2f15c22d068f1b2e2fc69f07","created_at":"2026-09-13T10:39:02.903Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.904Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2f15c22d068f1b2e2fc69f07: start","job_status":"start","queue_duration_s":0.153535,"scheduling_latency_s":0.15355} gitlab # [ 255.920135] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.075Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2f15c22d068f1b2e2fc69f07","created_at":"2026-09-13T10:39:02.903Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.904Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2f15c22d068f1b2e2fc69f07: done: 0.017444 sec","job_status":"done","queue_duration_s":0.153535,"scheduling_latency_s":0.15355,"redis_calls":7,"redis_duration_s":0.000904,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000659,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000245,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017444,"completed_at":"2026-09-13T10:39:03.075Z","load_balancing_strategy":"primary","db_duration_s":0.002702,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.932285] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.076Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"9a7de20da071331aaa61c176","created_at":"2026-09-13T10:39:02.909Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.911Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-9a7de20da071331aaa61c176: start","job_status":"start","queue_duration_s":0.165618,"scheduling_latency_s":0.165633} gitlab # [ 255.973592] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 57 gitlab # [ 255.974200] gitlab-workhorse[829]: time="2026-09-13T10:39:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 255.974659] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 69 gitlab # [ 255.975311] gitaly[918]: time="2026-09-13T10:39:03.174Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=11 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JQEEQRZAKG699D4R5P5B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:03.161" grpc.time_ms=12.899 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 255.976086] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.106Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"9a7de20da071331aaa61c176","created_at":"2026-09-13T10:39:02.909Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.911Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-9a7de20da071331aaa61c176: done: 0.029715 sec","job_status":"done","queue_duration_s":0.165618,"scheduling_latency_s":0.165633,"redis_calls":10,"redis_duration_s":0.003984,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002604,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000443,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029715,"completed_at":"2026-09-13T10:39:03.106Z","load_balancing_strategy":"primary","db_duration_s":0.003871,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.002982] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.107Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"75e78698179d37ae6982a864","created_at":"2026-09-13T10:39:02.911Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.912Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-75e78698179d37ae6982a864: start","job_status":"start","queue_duration_s":0.195191,"scheduling_latency_s":0.19521} gitlab # [ 256.043800] gitaly[918]: time="2026-09-13T10:39:03.192Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQEYW858B59CX2QMRXH7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:33.191" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:03.191" grpc.time_ms=0.519 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 256.044624] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.139Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"75e78698179d37ae6982a864","created_at":"2026-09-13T10:39:02.911Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.912Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-75e78698179d37ae6982a864: done: 0.03166 sec","job_status":"done","queue_duration_s":0.195191,"scheduling_latency_s":0.19521,"redis_calls":7,"redis_duration_s":0.003411,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002687,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000724,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03166,"completed_at":"2026-09-13T10:39:03.139Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009743,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.101737] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.142Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"84a3bc182c44949d54c9736a","created_at":"2026-09-13T10:39:02.912Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.914Z","job_size_bytes":19,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-84a3bc182c44949d54c9736a: start","job_status":"start","queue_duration_s":0.228255,"scheduling_latency_s":0.22829} gitlab # [ 256.102434] gitaly[918]: time="2026-09-13T10:39:03.204Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JQEYW858B59CX2QMRXH7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-13T10:39:33.197" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:03.197" grpc.time_ms=6.751 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 256.103227] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:03 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d2fb939f2bad435ffeda072a310aa498ab167846 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 258 gitlab # [ 256.103729] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.167Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"84a3bc182c44949d54c9736a","created_at":"2026-09-13T10:39:02.912Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.914Z","job_size_bytes":19,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-84a3bc182c44949d54c9736a: done: 0.024672 sec","job_status":"done","queue_duration_s":0.228255,"scheduling_latency_s":0.22829,"redis_calls":8,"redis_duration_s":0.0021809999999999998,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001364,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000817,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024672,"completed_at":"2026-09-13T10:39:03.167Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003338,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.170745] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.169Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c79d421a938c94e9a12a6de1","created_at":"2026-09-13T10:39:02.914Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.916Z","job_size_bytes":66,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c79d421a938c94e9a12a6de1: start","job_status":"start","queue_duration_s":0.252928,"scheduling_latency_s":0.25296} gitlab # [ 256.171485] gitaly[918]: time="2026-09-13T10:39:03.222Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JQEYW858B59CX2QMRXH7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:39:13.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:03.213" grpc.time_ms=9.164 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 256.172077] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.187Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c79d421a938c94e9a12a6de1","created_at":"2026-09-13T10:39:02.914Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.916Z","job_size_bytes":66,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c79d421a938c94e9a12a6de1: done: 0.018573 sec","job_status":"done","queue_duration_s":0.252928,"scheduling_latency_s":0.25296,"redis_calls":7,"redis_duration_s":0.002463,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00187,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000593,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018573,"completed_at":"2026-09-13T10:39:03.187Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002291,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.240887] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.191Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"00464800c133f8dee37a9ccd","created_at":"2026-09-13T10:39:02.916Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.919Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-00464800c133f8dee37a9ccd: start","job_status":"start","queue_duration_s":0.272149,"scheduling_latency_s":0.272167} gitlab # [ 256.241630] gitaly[918]: time="2026-09-13T10:39:03.246Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQGAJXA85VFKPGHRWEVM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:03.240" pid=918 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 256.242261] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.217Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"00464800c133f8dee37a9ccd","created_at":"2026-09-13T10:39:02.916Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.919Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-00464800c133f8dee37a9ccd: done: 0.026055 sec","job_status":"done","queue_duration_s":0.272149,"scheduling_latency_s":0.272167,"redis_calls":7,"redis_duration_s":0.00243,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002116,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000314,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026055,"completed_at":"2026-09-13T10:39:03.217Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005905,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.285585] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.218Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"6879d8806ade509d87d1d4a3","created_at":"2026-09-13T10:39:02.920Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.922Z","job_size_bytes":6,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-6879d8806ade509d87d1d4a3: start","job_status":"start","queue_duration_s":0.295983,"scheduling_latency_s":0.296001} gitlab # [ 256.286511] gitaly[918]: time="2026-09-13T10:39:03.246Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQGAJXA85VFKPGHRWEVM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:03.240" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 256.294562] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.232Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"6879d8806ade509d87d1d4a3","created_at":"2026-09-13T10:39:02.920Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.922Z","job_size_bytes":6,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-6879d8806ade509d87d1d4a3: done: 0.01358 sec","job_status":"done","queue_duration_s":0.295983,"scheduling_latency_s":0.296001,"redis_calls":7,"redis_duration_s":0.001487,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000977,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00051,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01358,"completed_at":"2026-09-13T10:39:03.232Z","load_balancing_strategy":"primary","db_duration_s":0.002344,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.336709] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.236Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"ab30bb57f1979640ec2ae7c4","created_at":"2026-09-13T10:39:02.923Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.926Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-ab30bb57f1979640ec2ae7c4: start","job_status":"start","queue_duration_s":0.309706,"scheduling_latency_s":0.309722} gitlab # [ 256.344959] gitaly[918]: time="2026-09-13T10:39:03.248Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQGAJXA85VFKPGHRWEVM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:39:03.240" grpc.time_ms=6.812 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846' gitlab # [ 256.387130] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.259Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"ab30bb57f1979640ec2ae7c4","created_at":"2026-09-13T10:39:02.923Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:02.926Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-ab30bb57f1979640ec2ae7c4: done: 0.023937 sec","job_status":"done","queue_duration_s":0.309706,"scheduling_latency_s":0.309722,"redis_calls":7,"redis_duration_s":0.004809,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004546,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011833,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023937,"completed_at":"2026-09-13T10:39:03.259Z","load_balancing_strategy":"primary","db_duration_s":0.005004,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.393103] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.261Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"d72b586c46756f1d5b02dc63","created_at":"2026-09-13T10:39:03.001Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.002Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d72b586c46756f1d5b02dc63: start","job_status":"start","queue_duration_s":0.258802,"scheduling_latency_s":0.258819} gitlab # [ 256.406851] gitaly[918]: time="2026-09-13T10:39:03.253Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQEYW858B59CX2QMRXH7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T10:39:13.043" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:39:03.253" grpc.time_ms=0.145 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 256.449708] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.472Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"d72b586c46756f1d5b02dc63","created_at":"2026-09-13T10:39:03.001Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-88671cc80d134ae7","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.002Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d72b586c46756f1d5b02dc63: done: 0.211973 sec","job_status":"done","queue_duration_s":0.258802,"scheduling_latency_s":0.258819,"gitaly_calls":3,"gitaly_duration_s":0.084794,"redis_calls":63,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.008675,"redis_read_bytes":144,"redis_write_bytes":16721,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000697,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000669,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":20,"redis_queues_duration_s":0.001729,"redis_queues_read_bytes":24,"redis_queues_write_bytes":11227,"redis_queues_metadata_calls":18,"redis_queues_metadata_duration_s":0.002911,"redis_queues_metadata_read_bytes":28,"redis_queues_metadata_write_bytes":2886,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000151,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000333,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001488,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.091215,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.183585903,"duration_s":0.211973,"completed_at":"2026-09-13T10:39:03.472Z","load_balancing_strategy":"primary","db_duration_s":0.028477,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 256.459932] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.474Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"4320a6a3a1bbfd83f8d96f34","created_at":"2026-09-13T10:39:03.315Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.317Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-4320a6a3a1bbfd83f8d96f34: start","job_status":"start","queue_duration_s":0.156306,"scheduling_latency_s":0.156324} gitlab # [ 256.508952] nixbot[2669]: {"ts": "2026-09-13T10:39:03.718Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d2fb939f2bad435ffeda072a310aa498ab167846 \"HTTP/1.1 201 Created\""} gitlab # [ 256.509615] nixbot[2669]: {"ts": "2026-09-13T10:39:03.729Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-d2fb939f2bad-4"} gitlab # [ 256.510407] gitaly[918]: time="2026-09-13T10:39:03.263Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQEYW858B59CX2QMRXH7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T10:39:13.047" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:03.258" grpc.time_ms=5.364 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 256.511083] gitaly[918]: time="2026-09-13T10:39:03.302Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JQEYW858B59CX2QMRXH7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:39:03.302" grpc.time_ms=0.322 pid=918 span.kind=server system=grpc gitlab # [ 256.511856] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.501Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"4320a6a3a1bbfd83f8d96f34","created_at":"2026-09-13T10:39:03.315Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.317Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-4320a6a3a1bbfd83f8d96f34: done: 0.0273 sec","job_status":"done","queue_duration_s":0.156306,"scheduling_latency_s":0.156324,"redis_calls":7,"redis_duration_s":0.000996,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000727,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000269,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019497,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0273,"completed_at":"2026-09-13T10:39:03.501Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005037,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 256.558210] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.502Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"0703c9cfb4932fce4701323e","created_at":"2026-09-13T10:39:03.339Z","trace_propagation_headers":{"sentry-trace":"47eb8aedf2f4427c8a6d5e4df48afc0f-a969933ac19b45bf","baggage":"sentry-trace_id=47eb8aedf2f4427c8a6d5e4df48afc0f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5JQEYW858B59CX2QMRXH7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.342Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-0703c9cfb4932fce4701323e: start","job_status":"start","queue_duration_s":0.160358,"scheduling_latency_s":0.160375} gitlab # [ 256.558864] gitaly[918]: time="2026-09-13T10:39:03.325Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JQEYW858B59CX2QMRXH7 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:39:03.325" grpc.time_ms=0.249 pid=918 span.kind=server system=grpc gitlab # [ 256.559456] gitaly[918]: time="2026-09-13T10:39:03.334Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQEYW858B59CX2QMRXH7 diskcache=71f4e452-a11b-4615-b2b6-2e8a65dc543d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:39:13.055" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:39:03.265" pid=918 ref=refs/keep-around/d2fb939f2bad435ffeda072a310aa498ab167846 remote_ip="2001:db8:1::1" revision=d2fb939f2bad435ffeda072a310aa498ab167846 span.kind=server system=grpc username=root gitlab # [ 256.599037] nixbot[2669]: {"ts": "2026-09-13T10:39:03.831Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-d2fb939f2bad-4", "argv0": "bwrap"} gitlab # [ 256.599699] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.523Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"0703c9cfb4932fce4701323e","created_at":"2026-09-13T10:39:03.339Z","trace_propagation_headers":{"sentry-trace":"47eb8aedf2f4427c8a6d5e4df48afc0f-a969933ac19b45bf","baggage":"sentry-trace_id=47eb8aedf2f4427c8a6d5e4df48afc0f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5JQEYW858B59CX2QMRXH7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.342Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-0703c9cfb4932fce4701323e: done: 0.020493 sec","job_status":"done","queue_duration_s":0.160358,"scheduling_latency_s":0.160375,"redis_calls":7,"redis_duration_s":0.0027749999999999997,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002486,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000289,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020493,"completed_at":"2026-09-13T10:39:03.523Z","load_balancing_strategy":"primary","db_duration_s":0.003008,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 256.605623] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.524Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"1b2974526bfe891b66455612","created_at":"2026-09-13T10:39:03.342Z","trace_propagation_headers":{"sentry-trace":"47eb8aedf2f4427c8a6d5e4df48afc0f-a969933ac19b45bf","baggage":"sentry-trace_id=47eb8aedf2f4427c8a6d5e4df48afc0f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5JQEYW858B59CX2QMRXH7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.344Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-1b2974526bfe891b66455612: start","job_status":"start","queue_duration_s":0.180021,"scheduling_latency_s":0.180036} gitlab # [ 256.621021] gitaly[918]: time="2026-09-13T10:39:03.334Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=50 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5316 command.oublock=8 command.real_time_ms=62 command.spawn_token_fork_ms=0 command.system_time_ms=36 command.user_time_ms=14 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQEYW858B59CX2QMRXH7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:39:13.055" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:39:03.265" grpc.time_ms=68.965 pid=918 ref=refs/keep-around/d2fb939f2bad435ffeda072a310aa498ab167846 remote_ip="2001:db8:1::1" revision=d2fb939f2bad435ffeda072a310aa498ab167846 span.kind=server system=grpc username=root gitlab # [ 256.661808] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.558Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"1b2974526bfe891b66455612","created_at":"2026-09-13T10:39:03.342Z","trace_propagation_headers":{"sentry-trace":"47eb8aedf2f4427c8a6d5e4df48afc0f-a969933ac19b45bf","baggage":"sentry-trace_id=47eb8aedf2f4427c8a6d5e4df48afc0f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5JQEYW858B59CX2QMRXH7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.344Z","job_size_bytes":65,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-1b2974526bfe891b66455612: done: 0.034285 sec","job_status":"done","queue_duration_s":0.180021,"scheduling_latency_s":0.180036,"redis_calls":10,"redis_duration_s":0.001362,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000773,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000589,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034285,"completed_at":"2026-09-13T10:39:03.558Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008835,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.700344] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.559Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"b8765a8ea10e02d281230355","created_at":"2026-09-13T10:39:03.379Z","trace_propagation_headers":{"sentry-trace":"47eb8aedf2f4427c8a6d5e4df48afc0f-a969933ac19b45bf","baggage":"sentry-trace_id=47eb8aedf2f4427c8a6d5e4df48afc0f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5JQEYW858B59CX2QMRXH7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.381Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-b8765a8ea10e02d281230355: start","job_status":"start","queue_duration_s":0.178152,"scheduling_latency_s":0.178171} gitlab # [ 256.701141] gitaly[918]: time="2026-09-13T10:39:03.338Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQEYW858B59CX2QMRXH7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:33.337" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:03.337" grpc.time_ms=0.379 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 256.701784] gitaly[918]: time="2026-09-13T10:39:03.345Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:39:03.344" grpc.time_ms=0.472 pid=918 span.kind=server system=grpc gitlab # [ 256.739020] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.635Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"b8765a8ea10e02d281230355","created_at":"2026-09-13T10:39:03.379Z","trace_propagation_headers":{"sentry-trace":"47eb8aedf2f4427c8a6d5e4df48afc0f-a969933ac19b45bf","baggage":"sentry-trace_id=47eb8aedf2f4427c8a6d5e4df48afc0f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5JQEYW858B59CX2QMRXH7","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.381Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-b8765a8ea10e02d281230355: done: 0.076485 sec","job_status":"done","queue_duration_s":0.178152,"scheduling_latency_s":0.178171,"gitaly_calls":2,"gitaly_duration_s":0.004207,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.003769,"redis_read_bytes":80,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000559,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000587,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000322,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000131,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000233,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000518,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055283,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.056916731,"duration_s":0.076485,"completed_at":"2026-09-13T10:39:03.635Z","load_balancing_strategy":"primary","db_duration_s":0.016258,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 256.777795] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.636Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"11db6c17094dbfd7cf006e78","created_at":"2026-09-13T10:39:03.406Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.408Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-11db6c17094dbfd7cf006e78: start","job_status":"start","queue_duration_s":0.228426,"scheduling_latency_s":0.228441} gitlab # [ 256.778664] gitaly[918]: time="2026-09-13T10:39:03.367Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:39:03.366" grpc.time_ms=0.99 pid=918 span.kind=server system=grpc gitlab # [ 256.779217] gitaly[918]: time="2026-09-13T10:39:03.389Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:39:03.388" grpc.time_ms=0.581 pid=918 span.kind=server system=grpc gitlab # [ 256.779867] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.653Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"11db6c17094dbfd7cf006e78","created_at":"2026-09-13T10:39:03.406Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.408Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-11db6c17094dbfd7cf006e78: done: 0.016471 sec","job_status":"done","queue_duration_s":0.228426,"scheduling_latency_s":0.228441,"redis_calls":7,"redis_duration_s":0.000819,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000605,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000214,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016471,"completed_at":"2026-09-13T10:39:03.653Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002391,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.829607] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.654Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"71f8ef37cf579749e952bf21","created_at":"2026-09-13T10:39:03.415Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.417Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-71f8ef37cf579749e952bf21: start","job_status":"start","queue_duration_s":0.237075,"scheduling_latency_s":0.237089} gitlab # [ 256.830413] gitaly[918]: time="2026-09-13T10:39:03.392Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:39:33.322" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:03.323" operations="[repack_refs]" pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 256.835687] gitaly[918]: time="2026-09-13T10:39:03.393Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:39:33.322" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:03.323" pid=918 remote_ip="2001:db8:1::1" repository_info="{false {6 704 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-13 10:38:03.920947053 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root gitlab # [ 256.873916] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.670Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"71f8ef37cf579749e952bf21","created_at":"2026-09-13T10:39:03.415Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.417Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-71f8ef37cf579749e952bf21: done: 0.016613 sec","job_status":"done","queue_duration_s":0.237075,"scheduling_latency_s":0.237089,"redis_calls":7,"redis_duration_s":0.00082,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000603,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000217,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016613,"completed_at":"2026-09-13T10:39:03.670Z","load_balancing_strategy":"primary","db_duration_s":0.00244,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.915598] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.671Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"19b5683baad6d61fe3a97bab","created_at":"2026-09-13T10:39:03.423Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.424Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-19b5683baad6d61fe3a97bab: start","job_status":"start","queue_duration_s":0.247104,"scheduling_latency_s":0.247119} gitlab # [ 256.916509] gitaly[918]: time="2026-09-13T10:39:03.400Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 diskcache=e96bc0f2-a48c-48de-9f40-749147dafdbb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:39:33.322" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:03.323" pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 256.917247] gitaly[918]: time="2026-09-13T10:39:03.400Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=68 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7496 command.oublock=0 command.real_time_ms=70 command.spawn_token_fork_ms=0 command.system_time_ms=50 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:39:33.322" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:03.323" grpc.time_ms=77.585 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 256.917950] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.689Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"19b5683baad6d61fe3a97bab","created_at":"2026-09-13T10:39:03.423Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.424Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-19b5683baad6d61fe3a97bab: done: 0.017779 sec","job_status":"done","queue_duration_s":0.247104,"scheduling_latency_s":0.247119,"redis_calls":10,"redis_duration_s":0.0011519999999999998,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000627,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000309,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000216,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017779,"completed_at":"2026-09-13T10:39:03.689Z","load_balancing_strategy":"primary","db_duration_s":0.002624,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.956469] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.690Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"bf2050a9d38d69fe6233aea6","created_at":"2026-09-13T10:39:03.424Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.425Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-bf2050a9d38d69fe6233aea6: start","job_status":"start","queue_duration_s":0.264467,"scheduling_latency_s":0.264481} gitlab # [ 256.998207] gitaly[918]: time="2026-09-13T10:39:03.414Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:39:33.322" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:03.323" pid=918 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root gitlab # [ 256.998419] gitaly[918]: time="2026-09-13T10:39:03.414Z" level=info msg="optimized repository" clean_stale_data_time=20.050314ms clean_work_trees_time="72.635µs" commit_graph_time="4.47µs" correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="3.911µs" pid=918 prune_time="3.352µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="5.308µs" storage_name=default system=housekeeping total_time=20.234415ms gitlab # [ 256.998909] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.705Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"bf2050a9d38d69fe6233aea6","created_at":"2026-09-13T10:39:03.424Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.425Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-bf2050a9d38d69fe6233aea6: done: 0.015009 sec","job_status":"done","queue_duration_s":0.264467,"scheduling_latency_s":0.264481,"redis_calls":7,"redis_duration_s":0.000801,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000592,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000209,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015009,"completed_at":"2026-09-13T10:39:03.705Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.022260] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.706Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"e81ba9c2aaccd079c404cc42","created_at":"2026-09-13T10:39:03.426Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.427Z","job_size_bytes":19,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-e81ba9c2aaccd079c404cc42: start","job_status":"start","queue_duration_s":0.278731,"scheduling_latency_s":0.278745} gitlab # [ 257.063354] gitaly[918]: time="2026-09-13T10:39:03.414Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:39:33.322" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:03.323" pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.063985] gitaly[918]: time="2026-09-13T10:39:03.441Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:39:33.440" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:03.440" grpc.time_ms=0.722 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.064686] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.722Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"e81ba9c2aaccd079c404cc42","created_at":"2026-09-13T10:39:03.426Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.427Z","job_size_bytes":19,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-e81ba9c2aaccd079c404cc42: done: 0.016252 sec","job_status":"done","queue_duration_s":0.278731,"scheduling_latency_s":0.278745,"redis_calls":8,"redis_duration_s":0.0030789999999999997,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000829,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00225,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016252,"completed_at":"2026-09-13T10:39:03.722Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002105,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.115240] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.724Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"284503d3e4390ca0804abe24","created_at":"2026-09-13T10:39:03.428Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.429Z","job_size_bytes":66,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-284503d3e4390ca0804abe24: start","job_status":"start","queue_duration_s":0.295385,"scheduling_latency_s":0.295401} gitlab # [ 257.115962] gitaly[918]: time="2026-09-13T10:39:03.461Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JPT5SG56EX47VFCSH0C1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:33.460" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:03.460" grpc.time_ms=0.484 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.119704] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.745Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"284503d3e4390ca0804abe24","created_at":"2026-09-13T10:39:03.428Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.429Z","job_size_bytes":66,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-284503d3e4390ca0804abe24: done: 0.021059 sec","job_status":"done","queue_duration_s":0.295385,"scheduling_latency_s":0.295401,"redis_calls":7,"redis_duration_s":0.002056,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001725,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000331,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012843,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021059,"completed_at":"2026-09-13T10:39:03.745Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002184,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.167515] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.747Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"34620434c6bae4d24478692a","created_at":"2026-09-13T10:39:03.429Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.430Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-34620434c6bae4d24478692a: start","job_status":"start","queue_duration_s":0.3166,"scheduling_latency_s":0.316615} gitlab # [ 257.206181] gitaly[918]: time="2026-09-13T10:39:03.607Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JQEYW858B59CX2QMRXH7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:39:33.606" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:03.606" grpc.time_ms=0.656 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.206887] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.777Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"34620434c6bae4d24478692a","created_at":"2026-09-13T10:39:03.429Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.430Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-34620434c6bae4d24478692a: done: 0.030552 sec","job_status":"done","queue_duration_s":0.3166,"scheduling_latency_s":0.316615,"redis_calls":7,"redis_duration_s":0.007378,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001823,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005555,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030552,"completed_at":"2026-09-13T10:39:03.777Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006138,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.247263] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.781Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"1ed7e3687b2103affd4a7b5d","created_at":"2026-09-13T10:39:03.431Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.431Z","job_size_bytes":6,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-1ed7e3687b2103affd4a7b5d: start","job_status":"start","queue_duration_s":0.349173,"scheduling_latency_s":0.349189} gitlab # [ 257.248052] gitaly[918]: time="2026-09-13T10:39:03.628Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JQEYW858B59CX2QMRXH7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:33.628" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:03.628" grpc.time_ms=0.427 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.248709] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.791Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"1ed7e3687b2103affd4a7b5d","created_at":"2026-09-13T10:39:03.431Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.431Z","job_size_bytes":6,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-1ed7e3687b2103affd4a7b5d: done: 0.010893 sec","job_status":"done","queue_duration_s":0.349173,"scheduling_latency_s":0.349189,"redis_calls":7,"redis_duration_s":0.001025,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000769,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000256,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010893,"completed_at":"2026-09-13T10:39:03.791Z","load_balancing_strategy":"primary","db_duration_s":0.001146,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.294726] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.792Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f3ded275200dbbe02590ea47","created_at":"2026-09-13T10:39:03.432Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.433Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f3ded275200dbbe02590ea47: start","job_status":"start","queue_duration_s":0.359769,"scheduling_latency_s":0.359781} gitlab # [ 257.296124] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.812Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f3ded275200dbbe02590ea47","created_at":"2026-09-13T10:39:03.432Z","trace_propagation_headers":{"sentry-trace":"e6eba9e049164e70863a5a3b06b6225a-ee67a7b3fa09446a","baggage":"sentry-trace_id=e6eba9e049164e70863a5a3b06b6225a,sentry-release=Unknown"},"correlation_id":"01M2D5JPT5SG56EX47VFCSH0C1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.433Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f3ded275200dbbe02590ea47: done: 0.019156 sec","job_status":"done","queue_duration_s":0.359769,"scheduling_latency_s":0.359781,"redis_calls":7,"redis_duration_s":0.00378,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003433,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000347,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019156,"completed_at":"2026-09-13T10:39:03.811Z","load_balancing_strategy":"primary","db_duration_s":0.002962,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.334674] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.816Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"c3880c862018065a9b10f1df","created_at":"2026-09-13T10:39:03.596Z","trace_propagation_headers":{"sentry-trace":"47eb8aedf2f4427c8a6d5e4df48afc0f-f421276903d34fe7","baggage":"sentry-trace_id=47eb8aedf2f4427c8a6d5e4df48afc0f,sentry-release=Unknown"},"correlation_id":"01M2D5JQEYW858B59CX2QMRXH7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.597Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c3880c862018065a9b10f1df: start","job_status":"start","queue_duration_s":0.218531,"scheduling_latency_s":0.218565} gitlab # [ 257.346972] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:03.847Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"c3880c862018065a9b10f1df","created_at":"2026-09-13T10:39:03.596Z","trace_propagation_headers":{"sentry-trace":"47eb8aedf2f4427c8a6d5e4df48afc0f-f421276903d34fe7","baggage":"sentry-trace_id=47eb8aedf2f4427c8a6d5e4df48afc0f,sentry-release=Unknown"},"correlation_id":"01M2D5JQEYW858B59CX2QMRXH7","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:03.597Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c3880c862018065a9b10f1df: done: 0.031137 sec","job_status":"done","queue_duration_s":0.218531,"scheduling_latency_s":0.218565,"redis_calls":7,"redis_duration_s":0.003656,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002804,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000852,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031137,"completed_at":"2026-09-13T10:39:03.847Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003416,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.434821] nixbot[2669]: INFO: ::ffff:127.0.0.1:33774 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d2fb939f2bad435ffeda072a310aa498ab167846', in 1.04 seconds) (finished: subtest: push delivers the webhook, in 49.63 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses gitlab # [ 257.478302] nix-daemon[2835]: accepted connection from pid 3691, user nixbot gitlab # [ 257.525221] gitaly[918]: time="2026-09-13T10:39:04.829Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JS2NNTGBSDMMY8FYXKP7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:34.827" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:04.828" grpc.time_ms=0.658 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.552927] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:04 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 67 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses, in 0.10 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses gitlab # [ 258.680972] gitaly[918]: time="2026-09-13T10:39:05.984Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JT5BBSVRP7V256G1A1FV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:35.483" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:05.983" grpc.time_ms=0.928 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.713697] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:06 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 117 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses, in 0.16 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses gitlab # [ 259.817406] gitaly[918]: time="2026-09-13T10:39:07.121Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JV9N1H294SXPGY57NA84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:37.119" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:07.120" grpc.time_ms=0.821 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 259.877255] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 118 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses, in 0.16 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses gitlab # [ 260.992975] gitaly[918]: time="2026-09-13T10:39:08.296Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JWE1ZPRBYZY5SQQFW62M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:38.295" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:08.295" grpc.time_ms=0.903 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 261.021769] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:08 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 91 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses, in 0.14 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses gitlab # [ 262.122811] gitaly[918]: time="2026-09-13T10:39:09.426Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JXHQFBRZPCDPGV5W5QQ9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:39.424" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:09.425" grpc.time_ms=0.926 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.151525] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:09 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 84 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses, in 0.13 seconds) gitlab # [ 262.480672] nix-daemon[2835]: accepted connection from pid 3732, user nixbot gitlab # [ 262.494101] nix-daemon[2835]: accepted connection from pid 3691, user nixbot gitlab # [ 262.591373] gitaly[918]: time="2026-09-13T10:39:09.896Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JY0H1754T6NJJBZ4AR3G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:39.394" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:09.895" grpc.time_ms=0.677 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.666814] gitaly[918]: time="2026-09-13T10:39:09.971Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JY0H1754T6NJJBZ4AR3G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:39:19.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:09.960" grpc.time_ms=10.713 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.693812] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:09.997Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4440a07dbb193321838f4bd5","created_at":"2026-09-13T10:39:09.993Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-a3e990c29c3d45ce","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:09.995Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4440a07dbb193321838f4bd5: start","job_status":"start","queue_duration_s":0.001839,"scheduling_latency_s":0.00187} gitlab # [ 262.707438] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:10 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d2fb939f2bad435ffeda072a310aa498ab167846 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 169 gitlab # [ 262.709531] nixbot[2669]: {"ts": "2026-09-13T10:39:10.012Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d2fb939f2bad435ffeda072a310aa498ab167846 \"HTTP/1.1 201 Created\""} gitlab # [ 262.762959] gitaly[918]: time="2026-09-13T10:39:10.067Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JY5ZQNWZ9Q2TWNNDXC3D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:40.066" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:10.066" grpc.time_ms=0.693 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.801971] gitaly[918]: time="2026-09-13T10:39:10.106Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JY5ZQNWZ9Q2TWNNDXC3D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:39:20.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:10.100" grpc.time_ms=6.251 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.803663] gitaly[918]: time="2026-09-13T10:39:10.108Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JY0H1754T6NJJBZ4AR3G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:39:10.108" grpc.time_ms=0.231 pid=918 span.kind=server system=grpc gitlab # [ 262.830943] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:10 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d2fb939f2bad435ffeda072a310aa498ab167846 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 119 gitlab # [ 262.833076] nixbot[2669]: {"ts": "2026-09-13T10:39:10.136Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d2fb939f2bad435ffeda072a310aa498ab167846 \"HTTP/1.1 201 Created\""} gitlab # [ 262.846019] gitaly[918]: time="2026-09-13T10:39:10.150Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JY0H1754T6NJJBZ4AR3G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:39:10.149" grpc.time_ms=1.166 pid=918 span.kind=server system=grpc gitlab # [ 262.874466] gitaly[918]: time="2026-09-13T10:39:10.179Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JYA37V4MQM2B8YCW6EF4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:40.177" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:10.178" grpc.time_ms=0.843 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.889404] gitaly[918]: time="2026-09-13T10:39:10.194Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D5JY0H1754T6NJJBZ4AR3G grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:39:10.193" grpc.time_ms=0.728 pid=918 span.kind=server system=grpc gitlab # [ 262.899297] gitaly[918]: time="2026-09-13T10:39:10.204Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JYA37V4MQM2B8YCW6EF4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:39:20.108" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:10.198" grpc.time_ms=5.623 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.920296] gitaly[918]: time="2026-09-13T10:39:10.224Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JY0H1754T6NJJBZ4AR3G diskcache=20865832-5789-4906-ba37-c71a9c69affc grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:39:40.086" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:10.086" pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.929433] gitaly[918]: time="2026-09-13T10:39:10.225Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=90 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7194 command.oublock=0 command.real_time_ms=132 command.spawn_token_fork_ms=0 command.system_time_ms=58 command.user_time_ms=32 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JY0H1754T6NJJBZ4AR3G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:39:40.086" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:39:10.086" grpc.time_ms=138.811 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.941918] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:10 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d2fb939f2bad435ffeda072a310aa498ab167846 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 87 gitlab # [ 262.942601] nixbot[2669]: {"ts": "2026-09-13T10:39:10.235Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d2fb939f2bad435ffeda072a310aa498ab167846 \"HTTP/1.1 201 Created\""} gitlab # [ 262.962253] gitaly[918]: time="2026-09-13T10:39:10.267Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D5JY0H1754T6NJJBZ4AR3G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:39:40.266" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:10.266" grpc.time_ms=0.711 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 262.991746] gitaly[918]: time="2026-09-13T10:39:10.296Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JY0H1754T6NJJBZ4AR3G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:40.295" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:10.295" grpc.time_ms=0.43 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.010946] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:10.315Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"069002f473c70a3f662cd762","created_at":"2026-09-13T10:39:10.313Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-f5294c138cbc4526","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"4440a07dbb193321838f4bd5","job_size_bytes":3,"pid":2556,"job_status":"deduplicated","message":"PipelineProcessWorker JID-069002f473c70a3f662cd762: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 263.011980] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:10.316Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4440a07dbb193321838f4bd5","created_at":"2026-09-13T10:39:09.993Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-a3e990c29c3d45ce","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:09.995Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2556,"job_status":"rescheduled","message":"PipelineProcessWorker JID-4440a07dbb193321838f4bd5: rescheduled"} gitlab # [ 263.016534] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:10.321Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4440a07dbb193321838f4bd5","created_at":"2026-09-13T10:39:09.993Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-a3e990c29c3d45ce","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:09.995Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4440a07dbb193321838f4bd5: done: 0.323502 sec","job_status":"done","queue_duration_s":0.001839,"scheduling_latency_s":0.00187,"gitaly_calls":3,"gitaly_duration_s":0.15634,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.012012,"redis_read_bytes":239,"redis_write_bytes":21778,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00118,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001028,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.002698,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14463,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.003913,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000357,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000356,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.0013,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.011,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.121879,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.278795769,"duration_s":0.323502,"completed_at":"2026-09-13T10:39:10.320Z","load_balancing_strategy":"primary","db_duration_s":0.040199,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 263.060966] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:10.322Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"61c50feacd63ae217420e877","created_at":"2026-09-13T10:39:10.080Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-f5294c138cbc4526","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:10.082Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-61c50feacd63ae217420e877: start","job_status":"start","queue_duration_s":0.239464,"scheduling_latency_s":0.23948} gitlab # [ 263.061848] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:10.347Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"61c50feacd63ae217420e877","created_at":"2026-09-13T10:39:10.080Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-f5294c138cbc4526","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:10.082Z","job_size_bytes":23,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-61c50feacd63ae217420e877: done: 0.025094 sec","job_status":"done","queue_duration_s":0.239464,"scheduling_latency_s":0.23948,"redis_calls":7,"redis_duration_s":0.00091,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00025,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018029,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025094,"completed_at":"2026-09-13T10:39:10.347Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004564,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 263.105137] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:10.348Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"55aca3c8300663e0a98b4887","created_at":"2026-09-13T10:39:10.229Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-f5294c138cbc4526","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:10.231Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-55aca3c8300663e0a98b4887: start","job_status":"start","queue_duration_s":0.116541,"scheduling_latency_s":0.116556} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses gitlab # [ 263.111031] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:10.366Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"55aca3c8300663e0a98b4887","created_at":"2026-09-13T10:39:10.229Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-f5294c138cbc4526","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:10.231Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-55aca3c8300663e0a98b4887: done: 0.018317 sec","job_status":"done","queue_duration_s":0.116541,"scheduling_latency_s":0.116556,"redis_calls":7,"redis_duration_s":0.001059,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018317,"completed_at":"2026-09-13T10:39:10.366Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002551,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.168398] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:10.367Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"ee592949ef53720f34a2a635","created_at":"2026-09-13T10:39:10.235Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-f5294c138cbc4526","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:10.238Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-ee592949ef53720f34a2a635: start","job_status":"start","queue_duration_s":0.12901,"scheduling_latency_s":0.12903} gitlab # [ 263.169079] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:10.384Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"ee592949ef53720f34a2a635","created_at":"2026-09-13T10:39:10.235Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-f5294c138cbc4526","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:10.238Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-ee592949ef53720f34a2a635: done: 0.01713 sec","job_status":"done","queue_duration_s":0.12901,"scheduling_latency_s":0.12903,"redis_calls":7,"redis_duration_s":0.000925,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000674,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01713,"completed_at":"2026-09-13T10:39:10.384Z","load_balancing_strategy":"primary","db_duration_s":0.00258,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.208257] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:10.385Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"08f159ebe47bf1edaafcc6b8","created_at":"2026-09-13T10:39:10.245Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-f5294c138cbc4526","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:10.246Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-08f159ebe47bf1edaafcc6b8: start","job_status":"start","queue_duration_s":0.138708,"scheduling_latency_s":0.138722} gitlab # [ 263.249141] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:10.403Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"08f159ebe47bf1edaafcc6b8","created_at":"2026-09-13T10:39:10.245Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-f5294c138cbc4526","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:10.246Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-08f159ebe47bf1edaafcc6b8: done: 0.017857 sec","job_status":"done","queue_duration_s":0.138708,"scheduling_latency_s":0.138722,"redis_calls":10,"redis_duration_s":0.0012799999999999999,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000646,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000385,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017857,"completed_at":"2026-09-13T10:39:10.403Z","load_balancing_strategy":"primary","db_duration_s":0.002827,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.252191] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:10.404Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d185bbd3ee35a647bb035b62","created_at":"2026-09-13T10:39:10.247Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-f5294c138cbc4526","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:10.248Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d185bbd3ee35a647bb035b62: start","job_status":"start","queue_duration_s":0.156022,"scheduling_latency_s":0.156036} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses, in 0.18 seconds) (finished: subtest: eval and build post commit statuses, in 5.88 seconds) (finished: run the VM test script, in 313.21 seconds) gitlab # [ 263.291313] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:10.422Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d185bbd3ee35a647bb035b62","created_at":"2026-09-13T10:39:10.247Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-f5294c138cbc4526","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:10.248Z","job_size_bytes":3,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d185bbd3ee35a647bb035b62: done: 0.01818 sec","job_status":"done","queue_duration_s":0.156022,"scheduling_latency_s":0.156036,"redis_calls":7,"redis_duration_s":0.0010969999999999999,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00083,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000267,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01818,"completed_at":"2026-09-13T10:39:10.422Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002494,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.332247] gitaly[918]: time="2026-09-13T10:39:10.592Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D5JYMVA2203HQ2N019MQP6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:39:40.591" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:39:10.592" grpc.time_ms=0.458 pid=918 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 263.333108] gitlab-workhorse[829]: gitlab 2001:db8:1::1 - - [2026/09/13:10:39:10 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d2fb939f2bad435ffeda072a310aa498ab167846/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 122 gitlab # [ 263.333508] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:10.423Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f0bd71d264acce0f09724658","created_at":"2026-09-13T10:39:10.248Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-f5294c138cbc4526","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:10.249Z","job_size_bytes":19,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f0bd71d264acce0f09724658: start","job_status":"start","queue_duration_s":0.173574,"scheduling_latency_s":0.173592} test script finished in 313.26s cleanup kill QemuMachine (pid 46) gitlab # [ 263.334230] sidekiq-cluster[2556]: {"severity":"INFO","time":"2026-09-13T10:39:10.438Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"f0bd71d264acce0f09724658","created_at":"2026-09-13T10:39:10.248Z","trace_propagation_headers":{"sentry-trace":"43b369733c5848e8846e2e64f3f55e2d-f5294c138cbc4526","baggage":"sentry-trace_id=43b369733c5848e8846e2e64f3f55e2d,sentry-release=Unknown"},"correlation_id":"01M2D5JY0H1754T6NJJBZ4AR3G","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-13T10:39:10.249Z","job_size_bytes":19,"pid":2556,"sidekiq_tid":"m7xk","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-f0bd71d264acce0f09724658: done: 0.014919 sec","job_status":"done","queue_duration_s":0.173574,"scheduling_latency_s":0.173592,"redis_calls":8,"redis_duration_s":0.0010119999999999999,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000236,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14) gitlab # r (finished: cleanup, in 1.42 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [] [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-13T10:38:14.115Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'd2fb939f2bad435ffeda072a310aa498ab167846', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-13T10:39:03.126981Z', 'started_at': '2026-09-13T10:39:03.724255Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}] [{'id': 5, 'sha': 'd2fb939f2bad435ffeda072a310aa498ab167846', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:39:03.375Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'd2fb939f2bad435ffeda072a310aa498ab167846', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:39:03.375Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'd2fb939f2bad435ffeda072a310aa498ab167846', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:39:03.375Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'd2fb939f2bad435ffeda072a310aa498ab167846', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:39:03.375Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'd2fb939f2bad435ffeda072a310aa498ab167846', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:39:03.375Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'd2fb939f2bad435ffeda072a310aa498ab167846', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 5s (1 warning)', 'created_at': '2026-09-13T10:39:03.375Z', 'started_at': None, 'finished_at': '2026-09-13T10:39:09.986Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'd2fb939f2bad435ffeda072a310aa498ab167846', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-09-13T10:39:10.120Z', 'started_at': None, 'finished_at': '2026-09-13T10:39:10.215Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]