vm-test-run-nixbot-gitlab
checks.x86_64-linux.nixbot-gitlab
· build #499
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.IEX9hE2r5G', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13(finished: start all VMs, in 54.88 seconds)14gitlab # Discarding device blocks: 0/262144 done15gitlab: waiting for unit setup-gitlab.service16gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes17gitlab: waiting for the VM to finish booting18gitlab # Filesystem UUID: 5190d224-4fcc-44e9-bb58-4d683bd603ed19gitlab # Superblock backups stored on blocks:20gitlab # 32768, 98304, 163840, 22937621gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/cp14g1dnxwabp53388g0brbl23fhly3x-nixos-system-gitlab-test/init regInfo=/nix/store/l3n6x225lvlzycphls65qqlim7a7agp8-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000001] kvm-clock: using sched offset of 472035542 cycles65gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000005] tsc: Detected 2400.012 MHz processor67gitlab # [ 0.000826] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000853] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000856] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.000894] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.002780] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.002793] Using GB pages for direct mapping73gitlab # [ 0.002867] RAMDISK: [mem 0xbe31c000-0xbffcffff]74gitlab # [ 0.002874] ACPI: Early table checksum verification disabled75gitlab # [ 0.002877] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.002882] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.002886] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002893] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002895] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.002896] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002898] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002900] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002901] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.002902] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.002903] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.002903] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.002904] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.002904] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003129] No NUMA configuration found90gitlab # [ 0.003130] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003132] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff]92gitlab # [ 0.005990] Zone ranges:93gitlab # [ 0.005991] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.005993] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.005995] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.005996] Device empty97gitlab # [ 0.005996] Movable zone start for each node98gitlab # [ 0.005997] Early memory node ranges99gitlab # [ 0.005998] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.005999] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.006000] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.006001] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.006022] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.006310] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.120024] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.120482] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.120497] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.120528] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.120530] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.120532] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.120533] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.120534] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.120535] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.120538] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.120539] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.120545] TSC deadline timer available117gitlab # [ 0.120550] CPU topo: Max. logical packages: 1118gitlab # [ 0.120551] CPU topo: Max. logical dies: 1119gitlab # [ 0.120551] CPU topo: Max. dies per package: 1120gitlab # [ 0.120555] CPU topo: Max. threads per core: 1121gitlab # [ 0.120556] CPU topo: Num. cores per package: 4122gitlab # [ 0.120556] CPU topo: Num. threads per package: 4123gitlab # [ 0.120556] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.120582] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.120600] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.120603] kvm-guest: setup PV sched yield127gitlab # [ 0.120619] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.120621] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.120622] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.120623] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.120625] Booting paravirtualized kernel on KVM132gitlab # [ 0.120629] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.125153] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.127576] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.127635] kvm-guest: PV spinlocks enabled136gitlab # [ 0.127637] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.127641] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/cp14g1dnxwabp53388g0brbl23fhly3x-nixos-system-gitlab-test/init regInfo=/nix/store/l3n6x225lvlzycphls65qqlim7a7agp8-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.127741] Unknown kernel command line parameters "regInfo=/nix/store/l3n6x225lvlzycphls65qqlim7a7agp8-closure-info/registration", will be passed to user space.139gitlab # [ 0.127754] random: crng init done140gitlab # [ 0.127755] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.137252] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.142179] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.142271] software IO TLB: area num 4.144gitlab # [ 0.222908] Fallback order for Node 0: 0145gitlab # [ 0.222917] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.222920] Policy zone: Normal147gitlab # [ 0.225662] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.236525] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.251276] allocated 12582912 bytes of page_ext150gitlab # [ 0.261672] ftrace: allocating 48717 entries in 192 pages151gitlab # [ 0.261675] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.262620] Dynamic Preempt: lazy153gitlab # [ 0.262813] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.262814] rcu: RCU event tracing is enabled.155gitlab # [ 0.262815] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.262817] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.262817] Rude variant of Tasks RCU enabled.158gitlab # [ 0.262818] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.262818] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.262819] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.262852] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.262853] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.262855] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.267411] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.267749] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.267756] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns167gitlab # [ 0.267876] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)168gitlab # [ 0.271729] Console: colour VGA+ 80x25169gitlab # [ 0.271733] printk: legacy console [tty0] enabled170gitlab # [ 0.303477] printk: legacy console [ttyS0] enabled171gitlab # [ 0.409998] ACPI: Core revision 20250807172gitlab # [ 0.410899] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns173gitlab # [ 0.412465] APIC: Switch to symmetric I/O mode setup174gitlab # [ 0.413508] x2apic enabled175gitlab # [ 0.414265] APIC: Switched APIC routing to: physical x2apic176gitlab # [ 0.415246] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()177gitlab # [ 0.416555] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()178gitlab # [ 0.418073] kvm-guest: setup PV IPIs179gitlab # [ 0.419773] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1180gitlab # [ 0.420862] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns181gitlab # [ 0.422673] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)182gitlab # [ 0.423769] x86/cpu: User Mode Instruction Prevention (UMIP) activated183gitlab # [ 0.425768] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127184gitlab # [ 0.427421] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0185gitlab # [ 0.428496] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto186gitlab # [ 0.429670] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl187gitlab # [ 0.431670] Transient Scheduler Attacks: Vulnerable: No microcode188gitlab # [ 0.432670] Spectre V2 : Mitigation: Enhanced / Automatic IBRS189gitlab # [ 0.433669] Speculative Return Stack Overflow: Mitigation: Safe RET190gitlab # [ 0.434671] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization191gitlab # [ 0.436680] Spectre V2 : Enabling IBPB for BPF192gitlab # [ 0.437670] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier193gitlab # [ 0.438671] active return thunk: srso_alias_return_thunk194gitlab # [ 0.439692] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'195gitlab # [ 0.441668] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'196gitlab # [ 0.442669] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'197gitlab # [ 0.443669] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'198gitlab # [ 0.444670] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'199gitlab # [ 0.445670] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'200gitlab # [ 0.446670] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'201gitlab # [ 0.448670] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256202gitlab # [ 0.449671] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64203gitlab # [ 0.450670] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512204gitlab # [ 0.451670] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024205gitlab # [ 0.452670] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8206gitlab # [ 0.453670] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.207gitlab # [ 0.485848] Freeing SMP alternatives memory: 44K208gitlab # [ 0.486673] pid_max: default: 32768 minimum: 301209gitlab # [ 0.487684] LSM: initializing lsm=capability,landlock,yama,bpf,ima210gitlab # [ 0.488786] landlock: Up and running.211gitlab # [ 0.489501] Yama: becoming mindful.212gitlab # [ 0.490384] LSM support for eBPF active213gitlab # [ 0.491433] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)214gitlab # [ 0.492827] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)215gitlab # [ 0.495810] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)216gitlab # [ 0.497326] Performance Events: Fam17h+ core perfctr, AMD PMU driver.217gitlab # [ 0.497680] ... version: 2218gitlab # [ 0.498555] ... bit width: 48219gitlab # [ 0.499263] ... generic counters: 6220gitlab # [ 0.499672] ... generic bitmap: 000000000000003f221gitlab # [ 0.500616] ... fixed-purpose counters: 0222gitlab # [ 0.501246] ... fixed-purpose bitmap: 0000000000000000223gitlab # [ 0.501672] ... value mask: 0000ffffffffffff224gitlab # [ 0.502673] ... max period: 00007fffffffffff225gitlab # [ 0.503672] ... global_ctrl mask: 000000000000003f226gitlab # [ 0.504786] signal: max sigframe size: 3376227gitlab # [ 0.505595] rcu: Hierarchical SRCU implementation.228gitlab # [ 0.506337] rcu: Max phase no-delay instances is 400.229gitlab # [ 0.506881] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level230gitlab # [ 0.512450] smp: Bringing up secondary CPUs ...231gitlab # [ 0.513772] smpboot: x86: Booting SMP configuration:232gitlab # [ 0.514683] .... node #0, CPUs: #1 #2 #3233gitlab # [ 0.517805] smp: Brought up 1 node, 4 CPUs234gitlab # [ 0.519673] smpboot: Total of 4 processors activated (19200.09 BogoMIPS)235gitlab # [ 0.521184] Memory: 6011548K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260108K reserved, 0K cma-reserved)236gitlab # [ 0.522673] devtmpfs: initialized237gitlab # [ 0.523752] x86/mm: Memory block size: 128MB238gitlab # [ 0.526861] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)239gitlab # [ 0.528782] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).240gitlab # [ 0.529782] pinctrl core: initialized pinctrl subsystem241gitlab # [ 0.531058] PM: RTC time: 09:19:15, date: 2026-11-12242gitlab # [ 0.534890] NET: Registered PF_NETLINK/PF_ROUTE protocol family243gitlab # [ 0.537056] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations244gitlab # [ 0.537742] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations245gitlab # [ 0.539842] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations246gitlab # [ 0.540685] audit: initializing netlink subsys (disabled)247gitlab # [ 0.541802] audit: type=2000 audit(1794475155.527:1): state=initialized audit_enabled=0 res=1248gitlab # [ 0.541966] thermal_sys: Registered thermal governor 'fair_share'249gitlab # [ 0.543672] thermal_sys: Registered thermal governor 'bang_bang'250gitlab # [ 0.544674] thermal_sys: Registered thermal governor 'step_wise'251gitlab # [ 0.545671] thermal_sys: Registered thermal governor 'user_space'252gitlab # [ 0.546673] thermal_sys: Registered thermal governor 'power_allocator'253gitlab # [ 0.547690] cpuidle: using governor menu254gitlab # [ 0.549920] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5255gitlab # [ 0.550921] PCI: Using configuration type 1 for base access256gitlab # [ 0.551673] PCI: Using configuration type 1 for extended access257gitlab # [ 0.552883] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.258gitlab # [ 0.555061] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages259gitlab # [ 0.556645] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page260gitlab # [ 0.557609] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages261gitlab # [ 0.558673] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page262gitlab # [ 0.563791] ACPI: Added _OSI(Module Device)263gitlab # [ 0.565682] ACPI: Added _OSI(Processor Device)264gitlab # [ 0.567678] ACPI: Added _OSI(Processor Aggregator Device)265gitlab # [ 0.574636] ACPI: 1 ACPI AML tables successfully acquired and loaded266gitlab # [ 0.580971] ACPI: Interpreter enabled267gitlab # [ 0.582411] ACPI: PM: (supports S0 S3 S4 S5)268gitlab # [ 0.582677] ACPI: Using IOAPIC for interrupt routing269gitlab # [ 0.583664] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug270gitlab # [ 0.584680] PCI: Using E820 reservations for host bridge windows271gitlab # [ 0.587630] ACPI: Enabled 2 GPEs in block 00 to 0F272gitlab # [ 0.601895] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])273gitlab # [ 0.604693] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]274gitlab # [ 0.610006] acpiphp: Slot [3] registered275gitlab # [ 0.611806] acpiphp: Slot [4] registered276gitlab # [ 0.613798] acpiphp: Slot [5] registered277gitlab # [ 0.614802] acpiphp: Slot [6] registered278gitlab # [ 0.616809] acpiphp: Slot [7] registered279gitlab # [ 0.618799] acpiphp: Slot [8] registered280gitlab # [ 0.619763] acpiphp: Slot [9] registered281gitlab # [ 0.621771] acpiphp: Slot [10] registered282gitlab # [ 0.622772] acpiphp: Slot [11] registered283gitlab # [ 0.624778] acpiphp: Slot [12] registered284gitlab # [ 0.625746] acpiphp: Slot [13] registered285gitlab # [ 0.627743] acpiphp: Slot [14] registered286gitlab # [ 0.628722] acpiphp: Slot [15] registered287gitlab # [ 0.630718] acpiphp: Slot [16] registered288gitlab # [ 0.631714] acpiphp: Slot [17] registered289gitlab # [ 0.632754] acpiphp: Slot [18] registered290gitlab # [ 0.634719] acpiphp: Slot [19] registered291gitlab # [ 0.635702] acpiphp: Slot [20] registered292gitlab # [ 0.636473] acpiphp: Slot [21] registered293gitlab # [ 0.637313] acpiphp: Slot [22] registered294gitlab # [ 0.637693] acpiphp: Slot [23] registered295gitlab # [ 0.638455] acpiphp: Slot [24] registered296gitlab # [ 0.639710] acpiphp: Slot [25] registered297gitlab # [ 0.640490] acpiphp: Slot [26] registered298gitlab # [ 0.641256] acpiphp: Slot [27] registered299gitlab # [ 0.641722] acpiphp: Slot [28] registered300gitlab # [ 0.642474] acpiphp: Slot [29] registered301gitlab # [ 0.643693] acpiphp: Slot [30] registered302gitlab # [ 0.644482] acpiphp: Slot [31] registered303gitlab # [ 0.644689] PCI host bridge to bus 0000:00304gitlab # [ 0.645471] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]305gitlab # [ 0.646591] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]306gitlab # [ 0.648723] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]307gitlab # [ 0.649673] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]308gitlab # [ 0.650672] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]309gitlab # [ 0.652690] pci_bus 0000:00: root bus resource [bus 00-ff]310gitlab # [ 0.655605] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint311gitlab # [ 0.660278] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint312gitlab # [ 0.664918] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint313gitlab # [ 0.673769] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]314gitlab # [ 0.675764] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk315gitlab # [ 0.678679] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk316gitlab # [ 0.680677] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk317gitlab # [ 0.683679] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk318gitlab # [ 0.686055] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint319gitlab # [ 0.691997] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]320gitlab # [ 0.696123] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint321gitlab # [ 0.699246] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI322gitlab # [ 0.701691] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB323gitlab # [ 0.703983] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint324gitlab # [ 0.720837] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]325gitlab # [ 0.723729] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]326gitlab # [ 0.725734] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]327gitlab # [ 0.729345] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]328gitlab # [ 0.734807] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint329gitlab # [ 0.745718] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]330gitlab # [ 0.748694] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]331gitlab # [ 0.751741] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]332gitlab # [ 0.779691] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]333gitlab # [ 0.784782] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint334gitlab # [ 0.791710] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]335gitlab # [ 0.792651] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]336gitlab # [ 0.793475] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]337gitlab # [ 0.797347] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint338gitlab # [ 0.801709] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]339gitlab # [ 0.802654] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]340gitlab # [ 0.803493] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]341gitlab # [ 0.806141] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint342gitlab # [ 0.810966] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]343gitlab # [ 0.812679] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]344gitlab # [ 0.813694] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]345gitlab # [ 0.815958] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint346gitlab # [ 0.819714] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]347gitlab # [ 0.820679] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]348gitlab # [ 0.822696] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]349gitlab # [ 0.824957] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint350gitlab # [ 0.828657] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]351gitlab # [ 0.829404] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]352gitlab # [ 0.830695] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]353gitlab # [ 0.833950] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint354gitlab # [ 0.836712] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]355gitlab # [ 0.837651] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]356gitlab # [ 0.839508] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]357gitlab # [ 0.840678] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]358gitlab # [ 0.843504] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint359gitlab # [ 0.845715] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]360gitlab # [ 0.846693] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]361gitlab # [ 0.848993] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint362gitlab # [ 0.852714] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]363gitlab # [ 0.854678] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]364gitlab # [ 0.855694] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]365gitlab # [ 0.857942] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint366gitlab # [ 0.860680] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]367gitlab # [ 0.861605] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]368gitlab # [ 0.863498] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]369gitlab # [ 0.868426] ACPI: PCI: Interrupt link LNKA configured for IRQ 10370gitlab # [ 0.869840] ACPI: PCI: Interrupt link LNKB configured for IRQ 10371gitlab # [ 0.870812] ACPI: PCI: Interrupt link LNKC configured for IRQ 11372gitlab # [ 0.871819] ACPI: PCI: Interrupt link LNKD configured for IRQ 11373gitlab # [ 0.873751] ACPI: PCI: Interrupt link LNKS configured for IRQ 9374gitlab # [ 0.876723] iommu: Default domain type: Translated375gitlab # [ 0.877580] iommu: DMA domain TLB invalidation policy: lazy mode376gitlab # [ 0.879016] ACPI: bus type USB registered377gitlab # [ 0.879982] usbcore: registered new interface driver usbfs378gitlab # [ 0.881726] usbcore: registered new interface driver hub379gitlab # [ 0.882668] usbcore: registered new device driver usb380gitlab # [ 0.884668] NetLabel: Initializing381gitlab # [ 0.885674] NetLabel: domain hash size = 128382gitlab # [ 0.886488] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO383gitlab # [ 0.887528] NetLabel: unlabeled traffic allowed by default384gitlab # [ 0.888444] PCI: Using ACPI for IRQ routing385gitlab # [ 0.890006] pci 0000:00:02.0: vgaarb: setting as boot VGA device386gitlab # [ 0.890668] pci 0000:00:02.0: vgaarb: bridge control possible387gitlab # [ 0.890668] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none388gitlab # [ 0.896687] vgaarb: loaded389gitlab # [ 0.899186] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0390gitlab # [ 0.900675] hpet0: 3 comparators, 64-bit 100.000000 MHz counter391gitlab # [ 0.906948] clocksource: Switched to clocksource kvm-clock392gitlab # [ 0.913284] VFS: Disk quotas dquot_6.6.0393gitlab # [ 0.915217] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)394gitlab # [ 0.919015] pnp: PnP ACPI init395gitlab # [ 0.921934] pnp: PnP ACPI: found 6 devices396gitlab # [ 0.936333] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns397gitlab # [ 0.940039] clocksource: Switched to clocksource acpi_pm398gitlab # [ 0.942589] NET: Registered PF_INET protocol family399gitlab # [ 0.947653] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)400gitlab # [ 0.977280] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)401gitlab # [ 0.978854] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)402gitlab # [ 0.980322] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)403gitlab # [ 0.984144] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)404gitlab # [ 0.985516] TCP: Hash tables configured (established 65536 bind 65536)405gitlab # [ 0.987614] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)406gitlab # [ 0.989291] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)407gitlab # [ 0.990794] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)408gitlab # [ 0.992251] NET: Registered PF_UNIX/PF_LOCAL protocol family409gitlab # [ 0.993335] NET: Registered PF_XDP protocol family410gitlab # [ 0.994244] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]411gitlab # [ 0.995341] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]412gitlab # [ 0.996419] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]413gitlab # [ 0.997595] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]414gitlab # [ 0.998771] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]415gitlab # [ 1.000197] pci 0000:00:01.0: PIIX3: Enabling Passive Release416gitlab # [ 1.001246] pci 0000:00:00.0: Limiting direct PCI/PCI transfers417gitlab # [ 1.003748] ACPI: \_SB_.LNKD: Enabled at IRQ 11418gitlab # [ 1.006450] PCI: CLS 0 bytes, default 64419gitlab # [ 1.007266] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)420gitlab # [ 1.007597] Trying to unpack rootfs image as initramfs...421gitlab # [ 1.008383] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB)422gitlab # [ 1.008511] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns423gitlab # [ 1.031975] Initialise system trusted keyrings424gitlab # [ 1.033319] workingset: timestamp_bits=40 max_order=21 bucket_order=0425gitlab # [ 1.045731] Key type asymmetric registered426gitlab # [ 1.046492] Asymmetric key parser 'x509' registered427gitlab # [ 1.047387] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)428gitlab # [ 1.049216] io scheduler mq-deadline registered429gitlab # [ 1.051400] io scheduler kyber registered430gitlab # [ 1.058773] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled431gitlab # [ 1.062396] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A432gitlab # [ 1.068572] Linux agpgart interface v0.103433gitlab # [ 1.070676] ACPI: bus type drm_connector registered434gitlab # [ 1.075619] usbcore: registered new interface driver usbserial_generic435gitlab # [ 1.078522] usbserial: USB Serial support registered for generic436gitlab # [ 1.081254] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.437gitlab # [ 1.081254] Please enable it if your BIOS has the CPPC option.438gitlab # [ 1.087103] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled439gitlab # [ 1.090899] drop_monitor: Initializing network drop monitor service440gitlab # [ 1.094046] NET: Registered PF_INET6 protocol family441gitlab # [ 1.098077] Segment Routing with IPv6442gitlab # [ 1.099525] In-situ OAM (IOAM) with IPv6443gitlab # [ 1.103409] IPI shorthand broadcast: enabled444gitlab # [ 1.111442] sched_clock: Marking stable (964018773, 146911936)->(1134347648, -23416939)445gitlab # [ 1.113431] registered taskstats version 1446gitlab # [ 1.114522] Loading compiled-in X.509 certificates447gitlab # [ 1.124408] Demotion targets for Node 0: null448gitlab # [ 1.125679] Key type .fscrypt registered449gitlab # [ 1.126450] Key type fscrypt-provisioning registered450gitlab # [ 1.127499] ima: No TPM chip found, activating TPM-bypass!451gitlab # [ 1.128485] ima: Allocated hash algorithm: sha1452gitlab # [ 1.129385] ima: No architecture policies found453gitlab # [ 1.130566] PM: Magic number: 2:158:323454gitlab # [ 1.131373] tty tty48: hash matches455gitlab # [ 1.133040] RAS: Correctable Errors collector initialized.456gitlab # [ 1.137581] clk: Disabling unused clocks457gitlab # [ 1.138395] PM: genpd: Disabling unused power domains458gitlab # [ 1.303613] Freeing initrd memory: 29392K459gitlab # [ 1.308096] Freeing unused decrypted memory: 2028K460gitlab # [ 1.310996] Freeing unused kernel image (initmem) memory: 3640K461gitlab # [ 1.312124] Write protecting the kernel read-only data: 32768k462gitlab # [ 1.314360] Freeing unused kernel image (text/rodata gap) memory: 1220K463gitlab # [ 1.317489] Freeing unused kernel image (rodata/data gap) memory: 760K464gitlab # [ 1.383805] x86/mm: Checked W+X mappings: passed, no W+X pages found.465gitlab # [ 1.385007] Run /init as init process466gitlab # [ 1.398537] systemd[1]: Inserted module 'autofs4'467gitlab # [ 1.421733] fuse: init (API version 7.45)468gitlab # [ 1.430224] ACPI: \_SB_.LNKC: Enabled at IRQ 10469gitlab # [ 1.437689] ACPI: \_SB_.LNKA: Enabled at IRQ 10470gitlab # [ 1.440939] ACPI: \_SB_.LNKB: Enabled at IRQ 11471gitlab # [ 1.479296] systemd[1]: Successfully made /usr/ read-only.472gitlab # [ 1.815564] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)473gitlab # [ 1.848642] systemd[1]: Detected virtualization kvm.474gitlab # [ 1.850885] systemd[1]: Detected architecture x86-64.475gitlab # [ 1.853121] systemd[1]: Running in initrd.476gitlab # [ 1.855672] systemd[1]: Initializing machine ID from random generator.477gitlab # [ 1.858711] systemd[1]: Hostname set to <gitlab>.478gitlab # [ 1.968601] systemd[1]: bpf-restrict-fs: LSM BPF program attached479gitlab # [ 2.014607] systemd[1]: Queued start job for default target Initrd Default Target.480gitlab # [ 2.025892] systemd[1]: Created slice Slice /system/modprobe.481gitlab # [ 2.027086] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.482gitlab # [ 2.028518] systemd[1]: Expecting device /dev/disk/by-label/nix-store...483gitlab # [ 2.029707] systemd[1]: Expecting device /dev/disk/by-label/nixos...484gitlab # [ 2.030866] systemd[1]: Reached target Path Units.485gitlab # [ 2.031751] systemd[1]: Reached target Slice Units.486gitlab # [ 2.032673] systemd[1]: Reached target Swaps.487gitlab # [ 2.033502] systemd[1]: Reached target Timer Units.488gitlab # [ 2.034521] systemd[1]: Listening on D-Bus System Message Bus Socket.489gitlab # [ 2.035784] systemd[1]: Listening on Journal Socket (/dev/log).490gitlab # [ 2.037010] systemd[1]: Listening on Journal Sockets.491gitlab # [ 2.038034] systemd[1]: Listening on udev Control Socket.492gitlab # [ 2.039120] systemd[1]: Listening on udev Kernel Socket.493gitlab # [ 2.040106] systemd[1]: Reached target Socket Units.494gitlab # [ 2.042262] systemd[1]: Starting Create List of Static Device Nodes...495gitlab # [ 2.044544] systemd[1]: Starting Load Kernel Module 9pnet_virtio...496gitlab # [ 2.046652] systemd[1]: Starting Load Kernel Module configfs...497gitlab # [ 2.050290] systemd[1]: Starting Journal Service...498gitlab # [ 2.057231] systemd[1]: Starting Load Kernel Modules...499gitlab # [ 2.058321] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os500gitlab # [ 2.069011] systemd[1]: Starting Coldplug All udev Devices...501gitlab # [ 2.072293] systemd-journald[91]: Collecting audit messages is disabled.502gitlab # [ 2.073963] systemd[1]: Finished Create List of Static Device Nodes.503gitlab # [ 2.075776] systemd[1]: modprobe@configfs.service: Deactivated successfully.504gitlab # [ 2.079278] systemd[1]: Finished Load Kernel Module configfs.505gitlab # [ 2.080708] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config506gitlab # [ 2.086490] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...507gitlab # [ 2.112009] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.508gitlab # [ 2.112114] netfs: FS-Cache loaded509gitlab # [ 2.113652] systemd[1]: Starting Create Static Device Nodes in /dev...510gitlab # [ 2.122110] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.511gitlab # [ 2.127154] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev512gitlab # [ 2.128987] systemd[1]: Finished Create Static Device Nodes in /dev.513gitlab # [ 2.129086] 9pnet: Installing 9P2000 support514gitlab # [ 2.129498] systemd[1]: Reached target Preparation for Local File Systems.515gitlab # [ 2.132257] systemd[1]: Reached target Local File Systems.516gitlab # [ 2.135003] systemd[1]: Starting Rule-based Manager for Device Events and Files...517gitlab # [ 2.138274] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.518gitlab # [ 2.140984] systemd[1]: Finished Load Kernel Module 9pnet_virtio.519gitlab # [ 2.147316] systemd[1]: Finished Load Kernel Modules.520gitlab # [ 2.000719] systemd-modules-load[94]: Using 4 probe threads521gitlab # [ 2.002754] systemd-modules-load[94]: Inserted module 'virtio_balloon'522gitlab # [ 2.149653] systemd[1]: Started Journal Service.523gitlab # [ 2.004073] systemd-modules-load[94]: Inserted module 'dm_mod'524gitlab # [ 2.005436] systemd-modules-load[94]: Inserted module 'virtio_gpu'525gitlab # [ 2.011898] systemd[1]: Starting Apply Kernel Variables...526gitlab # [ 2.015172] systemd[1]: Starting Create System Files and Directories...527gitlab # [ 2.022331] systemd-udevd[103]: Using default interface naming scheme 'v261'.528gitlab # [ 2.026064] systemd[1]: Finished Apply Kernel Variables.529gitlab # [ 2.028283] systemd[1]: Finished Create System Files and Directories.530gitlab # [ 2.038030] systemd[1]: Started Rule-based Manager for Device Events and Files.531gitlab # [ 2.068673] systemd[1]: Finished Coldplug All udev Devices.532gitlab # [ 2.069810] systemd[1]: Reached target System Initialization.533gitlab # [ 2.070709] systemd[1]: Reached target Basic System.534gitlab # [ 2.294608] SCSI subsystem initialized535gitlab # [ 2.296844] uhci_hcd 0000:00:01.2: UHCI Host Controller536gitlab # [ 2.301709] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1537gitlab # [ 2.303776] uhci_hcd 0000:00:01.2: detected 2 ports538gitlab # [ 2.305517] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140539gitlab # [ 2.307554] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18540gitlab # [ 2.308973] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1541gitlab # [ 2.310033] usb usb1: Product: UHCI Host Controller542gitlab # [ 2.310718] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd543gitlab # [ 2.311654] usb usb1: SerialNumber: 0000:00:01.2544gitlab # [ 2.314649] hub 1-0:1.0: USB hub found545gitlab # [ 2.315947] hub 1-0:1.0: 2 ports detected546gitlab # [ 2.318547] virtio_blk virtio4: 4/0/0 default/read/poll queues547gitlab # [ 2.322659] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)548gitlab # [ 2.351609] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12549gitlab # [ 2.353100] virtio_blk virtio5: 4/0/0 default/read/poll queues550gitlab # [ 2.354705] scsi host0: ata_piix551gitlab # [ 2.358220] serio: i8042 KBD port at 0x60,0x64 irq 1552gitlab # [ 2.359049] serio: i8042 AUX port at 0x60,0x64 irq 12553gitlab # [ 2.361243] scsi host1: ata_piix554gitlab # [ 2.361309] virtio_blk virtio5: [vdb] 11953448 512-byte logical blocks (6.12 GB/5.70 GiB)555gitlab # [ 2.361524] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0556gitlab # [ 2.365668] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0557gitlab # [ 2.240052] (udev-worker)[139]: Network interface NamePolicy= disabled on kernel command line.558gitlab # [ 2.242257] systemd[1]: Starting Virtual Console Setup...559gitlab # [ 2.246478] (udev-worker)[115]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.560gitlab # [ 2.248684] (udev-worker)[115]: Network interface NamePolicy= disabled on kernel command line.561gitlab # [ 2.263359] systemd-vconsole-setup[150]: Configuration of first virtual console was skipped, ignoring remaining ones.562gitlab # [ 2.265137] systemd[1]: Found device /dev/disk/by-label/nixos.563gitlab # [ 2.266124] systemd[1]: Reached target Initrd Root Device.564gitlab # [ 2.266906] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...565gitlab # [ 2.268167] systemd[1]: Finished Virtual Console Setup.566gitlab # [ 2.274279] systemd[1]: Found device /dev/disk/by-label/nix-store.567gitlab # [ 2.290784] systemd-fsck[157]: nixos: clean, 12/65536 files, 13019/262144 blocks568gitlab # [ 2.294571] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.569gitlab # [ 2.526517] ata2: found unknown device (class 0)570gitlab # [ 2.528259] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100571gitlab # [ 2.532578] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5572gitlab # [ 2.535841] usb 1-1: new full-speed USB device number 2 using uhci_hcd573gitlab # [ 2.569664] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0574gitlab # [ 2.601381] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray575gitlab # [ 2.634778] cdrom: Uniform CD-ROM driver Revision: 3.20576gitlab # [ 2.705471] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00577gitlab # [ 2.709002] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10578gitlab # [ 2.711652] usb 1-1: Product: QEMU USB Tablet579gitlab # [ 2.713316] usb 1-1: Manufacturer: QEMU580gitlab # [ 2.714835] usb 1-1: SerialNumber: 28754-0000:00:01.2-1581gitlab # [ 2.748101] hid: raw HID events driver (C) Jiri Kosina582gitlab # [ 2.762084] usbcore: registered new interface driver usbhid583gitlab # [ 2.764240] usbhid: USB HID core driver584gitlab # [ 2.774492] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2585gitlab # [ 2.778675] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0586gitlab # [ 2.927391] systemd[1]: Mounting /sysroot...587gitlab # [ 3.233778] EXT4-fs (vda): mounted filesystem 5190d224-4fcc-44e9-bb58-4d683bd603ed r/w with ordered data mode. Quota mode: none.588gitlab # [ 3.091380] systemd[1]: Mounted /sysroot.589gitlab # [ 3.093287] systemd[1]: Reached target Initrd Root File System.590gitlab # [ 3.095708] systemd[1]: Mounting /sysroot/nix/.ro-store...591gitlab # [ 3.097599] systemd[1]: Mounting /sysroot/nix/.rw-store...592gitlab # [ 3.099702] systemd[1]: Mounting /sysroot/run...593gitlab # [ 3.104077] systemd[1]: Mounting /sysroot/tmp/shared...594gitlab # [ 3.111464] systemd[1]: Mounting /sysroot/tmp/xchg...595gitlab # [ 3.114347] systemd[1]: Starting Mountpoints Configured in the Real Root...596gitlab # [ 3.267019] 9p: Installing v9fs 9p2000 file system support597gitlab # [ 3.125725] systemd[1]: Mounted /sysroot/nix/.rw-store.598gitlab # [ 3.128259] systemd[1]: Mounted /sysroot/run.599gitlab # [ 3.130142] systemd[1]: Mounted /sysroot/tmp/shared.600gitlab # [ 3.135138] systemd[1]: Starting rw-sysroot-nix-store.service...601gitlab # [ 3.137515] systemd-sysroot-fstab-check[178]: /sysroot should be mounted in the initrd, will request daemon-reload.602gitlab # [ 3.140566] systemd[1]: Reload requested from client PID 178 ('systemd-sysroot') (unit initrd-parse-etc.service)...603gitlab # [ 3.143228] systemd[1]: Reloading...604gitlab # [ 3.295848] erofs (device vdb): mounted with root inode @ nid 128.605gitlab # [ 3.202961] systemd[1]: Reloading finished in 61 ms.606gitlab # [ 3.223726] systemd-sysroot-fstab-check[178]: Requesting initrd-fs.target/start/replace...607gitlab # [ 3.226859] systemd[1]: Mounted /sysroot/nix/.ro-store.608gitlab # [ 3.229070] systemd[1]: Mounted /sysroot/tmp/xchg.609gitlab # [ 3.230862] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.610gitlab # [ 3.233639] systemd[1]: Finished rw-sysroot-nix-store.service.611gitlab # [ 3.235732] systemd-sysroot-fstab-check[178]: Requesting swap.target/start/replace...612gitlab # [ 3.238403] systemd[1]: Starting rw-sysroot-nix-store.service...613gitlab # [ 3.240483] systemd[1]: initrd-parse-etc.service: Deactivated successfully.614gitlab # [ 3.242907] systemd[1]: Finished Mountpoints Configured in the Real Root.615gitlab # [ 3.245597] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.616gitlab # [ 3.255189] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.617gitlab # [ 3.258106] systemd[1]: Finished rw-sysroot-nix-store.service.618gitlab # [ 3.927666] systemd[1]: Mounting /sysroot/nix/store...619gitlab # [ 3.977422] systemd[1]: Mounted /sysroot/nix/store.620gitlab # [ 3.979535] systemd[1]: Reached target Initrd File Systems.621gitlab # [ 3.981600] systemd[1]: Starting Find NixOS closure...622gitlab # [ 3.983507] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...623gitlab # [ 4.018153] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.624gitlab # [ 4.020984] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.625gitlab # [ 4.041090] systemd[1]: Finished Find NixOS closure.626gitlab # [ 4.043122] systemd[1]: Reached target Initrd Default Target.627gitlab # [ 4.045152] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...628gitlab # [ 4.076459] systemd[1]: Stopped target Initrd Default Target.629gitlab # [ 4.078710] systemd[1]: Stopped target Basic System.630gitlab # [ 4.080710] systemd[1]: Stopped target Initrd Root Device.631gitlab # [ 4.082778] systemd[1]: Stopped target Path Units.632gitlab # [ 4.084651] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.633gitlab # [ 4.087367] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.634gitlab # [ 4.090109] systemd[1]: Stopped target Slice Units.635gitlab # [ 4.092561] systemd[1]: Stopped target Socket Units.636gitlab # [ 4.094492] systemd[1]: Stopped target System Initialization.637gitlab # [ 4.097908] systemd[1]: Stopped target Swaps.638gitlab # [ 4.101925] systemd[1]: Stopped target Timer Units.639gitlab # [ 4.103902] systemd[1]: dbus.socket: Deactivated successfully.640gitlab # [ 4.106122] systemd[1]: Closed D-Bus System Message Bus Socket.641gitlab # [ 4.108429] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.642gitlab # [ 4.111086] systemd[1]: Stopped Find NixOS closure.643gitlab # [ 4.112851] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio644gitlab # [ 4.116183] systemd[1]: Starting rw-sysroot-nix-store.service...645gitlab # [ 4.117934] systemd[1]: systemd-sysctl.service: Deactivated successfully.646gitlab # [ 4.119827] systemd[1]: Stopped Apply Kernel Variables.647gitlab # [ 4.121382] systemd[1]: systemd-modules-load.service: Deactivated successfully.648gitlab # [ 4.123374] systemd[1]: Stopped Load Kernel Modules.649gitlab # [ 4.124847] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.650gitlab # [ 4.126987] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.651gitlab # [ 4.129140] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.652gitlab # [ 4.131099] systemd[1]: Stopped Create System Files and Directories.653gitlab # [ 4.132808] systemd[1]: Stopped target Local File Systems.654gitlab # [ 4.134060] systemd[1]: Stopped target Preparation for Local File Systems.655gitlab # [ 4.135039] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.656gitlab # [ 4.136097] systemd[1]: Stopped Coldplug All udev Devices.657gitlab # [ 4.136976] systemd[1]: Stopping Rule-based Manager for Device Events and Files...658gitlab # [ 4.138077] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.659gitlab # [ 4.139111] systemd[1]: Stopped Virtual Console Setup.660gitlab # [ 4.139803] systemd[1]: initrd-cleanup.service: Deactivated successfully.661gitlab # [ 4.140820] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.662gitlab # [ 4.141759] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.663gitlab # [ 4.142809] systemd[1]: Finished rw-sysroot-nix-store.service.664gitlab # [ 4.143662] systemd[1]: systemd-udevd.service: Deactivated successfully.665gitlab # [ 4.144614] systemd[1]: Stopped Rule-based Manager for Device Events and Files.666gitlab # [ 4.145645] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.667gitlab # [ 4.146708] systemd[1]: Closed udev Control Socket.668gitlab # [ 4.147478] systemd[1]: Starting Cleanup udev Database...669gitlab # [ 4.148323] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.670gitlab # [ 4.149435] systemd[1]: Stopped Create Static Device Nodes in /dev.671gitlab # [ 4.150353] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.672gitlab # [ 4.151500] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.673gitlab # [ 4.152553] systemd[1]: kmod-static-nodes.service: Deactivated successfully.674gitlab # [ 4.153524] systemd[1]: Stopped Create List of Static Device Nodes.675gitlab # [ 4.157980] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.676gitlab # [ 4.159151] systemd[1]: Finished Cleanup udev Database.677gitlab # [ 4.160323] systemd[1]: Reached target Switch Root.678gitlab # [ 4.161633] systemd[1]: Starting NixOS Activation...679gitlab # [ 4.291520] initrd-nixos-activation-start[227]: booting system configuration /nix/store/cp14g1dnxwabp53388g0brbl23fhly3x-nixos-system-gitlab-test680gitlab # [ 4.341817] initrd-nixos-activation-start[227]: running activation script...681gitlab # [ 4.636125] initrd-nixos-activation-start[250]: setting up /etc...682gitlab # [ 4.748790] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.683gitlab # [ 4.751234] systemd[1]: Finished NixOS Activation.684gitlab # [ 4.753223] systemd[1]: Starting Switch Root...685gitlab # [ 4.792522] systemd[1]: Switching root.686gitlab # [ 4.997403] systemd-journald[91]: Received SIGTERM from PID 1 (systemd).687gitlab # [ 5.132268] NET: Registered PF_VSOCK protocol family688gitlab # [ 5.505481] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)689gitlab # [ 5.514668] systemd[1]: Detected virtualization kvm.690gitlab # [ 5.516507] systemd[1]: Detected architecture x86-64.691gitlab # [ 5.518528] systemd[1]: Detected first boot.692gitlab # [ 5.521185] systemd[1]: Initializing machine ID from random generator.693gitlab # [ 5.687197] systemd[1]: bpf-restrict-fs: LSM BPF program attached694gitlab # [ 5.793872] systemd[1]: Applying preset policy.695gitlab # [ 5.941351] systemd[1]: Populated /etc with preset unit settings.696gitlab # [ 6.053594] systemd[1]: initrd-switch-root.service: Deactivated successfully.697gitlab # [ 6.055045] systemd[1]: Stopped initrd-switch-root.service.698gitlab # [ 6.057412] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.699gitlab # [ 6.059836] systemd[1]: Created slice Slice /system/getty.700gitlab # [ 6.061260] systemd[1]: Created slice GitLab DevOps Platform Slice.701gitlab # [ 6.062750] systemd[1]: Created slice User and Session Slice.702gitlab # [ 6.063745] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.703gitlab # [ 6.064994] systemd[1]: Started Forward Password Requests to Wall Directory Watch.704gitlab # [ 6.066123] systemd[1]: Expecting device /dev/hvc0...705gitlab # [ 6.066844] systemd[1]: Expecting device /dev/ttyS0...706gitlab # [ 6.067659] systemd[1]: Reached target Local Encrypted Volumes.707gitlab # [ 6.068513] systemd[1]: Stopped target initrd-fs.target.708gitlab # [ 6.069284] systemd[1]: Stopped target initrd-root-fs.target.709gitlab # [ 6.070140] systemd[1]: Stopped target initrd-switch-root.target.710gitlab # [ 6.071127] systemd[1]: Reached target Virtual Machines and Containers.711gitlab # [ 6.072108] systemd[1]: Reached target Path Units.712gitlab # [ 6.072805] systemd[1]: Reached target Remote File Systems.713gitlab # [ 6.073619] systemd[1]: Reached target Slice Units.714gitlab # [ 6.074398] systemd[1]: Reached target Swaps.715gitlab # [ 6.075931] systemd[1]: Listening on Query the User Interactively for a Password.716gitlab # [ 6.077870] systemd[1]: Listening on Process Core Dump Socket.717gitlab # [ 6.079382] systemd[1]: Listening on Credential Encryption/Decryption.718gitlab # [ 6.081041] systemd[1]: Listening on Factory Reset Management.719gitlab # [ 6.081995] systemd[1]: Listening on Hostname Service Socket.720gitlab # [ 6.084452] systemd[1]: Starting Journal Log Access Socket...721gitlab # [ 6.086175] systemd[1]: Listening on Journal Audit Socket.722gitlab # [ 6.088135] systemd[1]: Listening on Console Output Muting Service Socket.723gitlab # [ 6.089298] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.724gitlab # [ 6.090487] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os725gitlab # [ 6.092036] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki726gitlab # [ 6.094954] systemd[1]: Listening on Disk Repartitioning Service Socket.727gitlab # [ 6.096026] systemd[1]: Listening on udev Control Socket.728gitlab # [ 6.096936] systemd[1]: Listening on udev Varlink Socket.729gitlab # [ 6.100027] systemd[1]: Mounting Huge Pages File System...730gitlab # [ 6.102162] systemd[1]: Mounting POSIX Message Queue File System...731gitlab # [ 6.104440] systemd[1]: Mounting Kernel Debug File System...732gitlab # [ 6.118233] systemd[1]: Mounting Kernel Trace File System...733gitlab # [ 6.124948] systemd[1]: Starting Create List of Static Device Nodes...734gitlab # [ 6.127742] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio735gitlab # [ 6.134271] systemd[1]: Starting Load Kernel Module configfs...736gitlab # [ 6.136527] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm737gitlab # [ 6.169188] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore738gitlab # [ 6.172402] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse739gitlab # [ 6.177524] systemd[1]: Mounting FUSE Control File System...740gitlab # [ 6.179327] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67741gitlab # [ 6.189910] systemd[1]: Starting Journal Service...742gitlab # [ 6.192052] systemd[1]: Starting Load Kernel Modules...743gitlab # [ 6.194639] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...744gitlab # [ 6.198973] systemd[1]: Starting Remount Root and Kernel File Systems...745gitlab # [ 6.200068] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os746gitlab # [ 6.202991] systemd[1]: Starting Coldplug All udev Devices...747gitlab # [ 6.205814] systemd[1]: Listening on Journal Log Access Socket.748gitlab # [ 6.213517] systemd-journald[320]: Collecting audit messages is enabled.749gitlab # [ 6.213629] systemd[1]: Mounted Huge Pages File System.750gitlab # [ 6.218427] systemd[1]: Mounted POSIX Message Queue File System.751gitlab # [ 6.219932] EXT4-fs (vda): re-mounted 5190d224-4fcc-44e9-bb58-4d683bd603ed.752gitlab # [ 6.221196] systemd[1]: Mounted Kernel Debug File System.753gitlab # [ 6.226036] systemd[1]: Mounted Kernel Trace File System.754gitlab # [ 6.082667] systemd[1]: Queued start job for default target Multi-User System.755gitlab # [ 6.083971] systemd[1]: Unnecessary job was removed for /dev/[ 6.230573] systemd[1]: Finished Create List of Static Device Nodes.756gitlab # vdb.757gitlab # [ 6.086939] systemd[1]: systemd-journald.service: Deactivated successfully.758gitlab # [ 6.234237] systemd[1]: Started Journal Service.759gitlab # [ 6.088470] systemd-modules-load[321]: Using 3 probe threads760gitlab # [ 6.090405] systemd[1]: modprobe@configfs.service: Deactivated successfully.761gitlab # [ 6.091573] systemd[1]: Finished Load Kernel Module configfs.762gitlab # [ 6.092639] systemd[1]: Mounted FUSE Control File System.763gitlab # [ 6.240130] loop: module loaded764gitlab # [ 6.093699] systemd[1]: Finished Remount Root and Kernel File Systems.765gitlab # [ 6.095042] systemd-modules-load[321]: Inserted module 'loop'766gitlab # [ 6.096412] systemd-modules-load[321]: Inserted module 'tls'767gitlab # [ 6.098699] systemd[1]: Finished Load Kernel Modules.768gitlab # [ 6.101643] systemd-oomd[322]: No swap; memory pressure usage will be degraded769gitlab # [ 6.103548] systemd[1]: Listening on Disk Image Download Service Socket.770gitlab # [ 6.104909] systemd[1]: Mounting Kernel Configuration File System...771gitlab # [ 6.107171] systemd[1]: Starting Firewall...772gitlab # [ 6.109153] systemd[1]: Starting Flush Journal to Persistent Storage...773gitlab # [ 6.110171] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore774gitlab # [ 6.114150] systemd[1]: Starting Load/Save OS Random Seed...775gitlab # [ 6.118605] systemd[1]: Starting Apply Kernel Variables...776gitlab # [ 6.121715] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...777gitlab # [ 6.274053] systemd-journald[320]: Received client request to flush runtime journal.778gitlab # [ 6.169456] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os779gitlab # [ 6.171369] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.780gitlab # [ 6.173512] systemd[1]: Mounted Kernel Configuration File System.781gitlab # [ 6.174589] systemd[1]: Finished Apply Kernel Variables.782gitlab # [ 6.176422] systemd[1]: Finished Load/Save OS Random Seed.783gitlab # [ 6.177374] systemd[1]: Reached target First Boot Complete.784gitlab # [ 6.179084] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.785gitlab # [ 6.180190] systemd[1]: Starting Create Static Device Nodes in /dev...786gitlab # [ 6.181134] systemd[1]: Finished Flush Journal to Persistent Storage.787gitlab # [ 6.182098] systemd[1]: Finished Create Static Device Nodes in /dev.788gitlab # [ 6.182963] systemd[1]: Reached target Preparation for Local File Systems.789gitlab # [ 6.184035] systemd[1]: Starting Rule-based Manager for Device Events and Files...790gitlab # [ 6.209711] systemd-udevd[367]: Using default interface naming scheme 'v261'.791gitlab # [ 6.231449] systemd[1]: Finished Coldplug All udev Devices.792gitlab # [ 6.241635] systemd[1]: Started Rule-based Manager for Device Events and Files.793gitlab # [ 6.315032] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse794gitlab # [ 6.341724] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.795gitlab # [ 6.343337] (udev-worker)[440]: Network interface NamePolicy= disabled on kernel command line.796gitlab # [ 6.346315] (udev-worker)[439]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.797gitlab # [ 6.348666] (udev-worker)[439]: Network interface NamePolicy= disabled on kernel command line.798gitlab # [ 6.355622] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.799gitlab # [ 6.366218] systemd[1]: Condition check resulted in Virtio network device being skipped.800gitlab # [ 6.368473] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore801gitlab # [ 6.370532] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67802gitlab # [ 6.372803] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore803gitlab # [ 6.374560] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os804gitlab # [ 6.375909] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os805gitlab # [ 6.378334] systemd[1]: Finished Firewall.806gitlab # [ 6.572686] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3807gitlab # [ 6.578028] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console808gitlab # [ 6.578821] mousedev: PS/2 mouse device common for all mice809gitlab # [ 6.578860] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0810gitlab # [ 6.578970] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD811gitlab # [ 6.585127] ACPI: button: Power Button [PWRF]812gitlab # [ 6.593815] Console: switching to colour dummy device 80x25813gitlab # [ 6.595111] [drm] Found bochs VGA, ID 0xb0c5.814gitlab # [ 6.595113] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.815gitlab # [ 6.605967] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic816gitlab # [ 6.606034] rtc_cmos 00:05: RTC can wake from S4817gitlab # [ 6.606749] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0818gitlab # [ 6.609235] parport_pc 00:03: reported by Plug and Play ACPI819gitlab # [ 6.609934] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]820gitlab # [ 6.610535] Floppy drive(s): fd0 is 2.88M AMI BIOS821gitlab # [ 6.612559] rtc_cmos 00:05: registered as rtc0822gitlab # [ 6.612628] rtc_cmos 00:05: setting system clock to 2026-11-12T09:19:21 UTC (1794475161)823gitlab # [ 6.612925] systemd-journald[320]: Time jumped backwards, rotating.824gitlab # [ 6.617039] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs825gitlab # [ 6.624126] Console: switching to colour frame buffer device 160x50826gitlab # [ 6.625500] FDC 0 is a S82078B827gitlab # [ 6.632045] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device828gitlab # [ 6.636157] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4829gitlab # [ 6.642058] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6830gitlab # [ 6.667290] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5831gitlab # [ 6.556301] systemd[1]: Starting Virtual Console Setup...832gitlab # [ 6.567172] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.833gitlab # [ 6.568552] systemd[1]: Stopped Virtual Console Setup.834gitlab # [ 6.569588] systemd[1]: Starting Virtual Console Setup...835gitlab # [ 6.600607] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.836gitlab # [ 6.601881] systemd[1]: Stopped Virtual Console Setup.837gitlab # [ 6.611869] systemd[1]: Starting Virtual Console Setup...838gitlab # [ 6.768396] ppdev: user-space parallel port driver839gitlab # [ 6.789725] kvm_amd: TSC scaling supported840gitlab # [ 6.790425] kvm_amd: Nested Virtualization enabled841gitlab # [ 6.791081] kvm_amd: Nested Paging enabled842gitlab # [ 6.791556] kvm_amd: LBR virtualization supported843gitlab # [ 6.792295] kvm_amd: Virtual VMLOAD VMSAVE supported844gitlab # [ 6.793020] kvm_amd: Virtual GIF supported845gitlab # [ 6.793529] kvm_amd: Virtual NMI enabled846gitlab # [ 6.828040] EDAC MC: Ver: 3.0.0847gitlab # [ 6.688266] systemd-vconsole-setup[495]: Configuration of first virtual console was skipped, ignoring remaining ones.848gitlab # [ 6.690911] systemd[1]: Finished Virtual Console Setup.849gitlab # [ 6.912603] systemd[1]: Mounting /run/wrappers...850gitlab # [ 6.940167] systemd[1]: Mounted /run/wrappers.851gitlab # [ 6.942149] systemd[1]: Reached target Local File Systems.852gitlab # [ 6.944160] systemd[1]: Listening on Boot Loader Control Service Socket.853gitlab # [ 6.946504] systemd[1]: Starting register-nix-paths.service...854gitlab # [ 6.948645] systemd[1]: Starting Create SUID/SGID Wrappers...855gitlab # [ 6.950866] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.856gitlab # [ 6.953981] systemd[1]: Starting Save Transient machine-id to Disk...857gitlab # [ 6.956962] systemd[1]: Starting Create System Files and Directories...858gitlab # [ 6.988646] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.859gitlab # [ 6.994425] systemd[1]: Finished Save Transient machine-id to Disk.860gitlab # [ 7.002586] systemd[1]: Finished Create System Files and Directories.861gitlab # [ 7.006450] systemd[1]: Starting Rebuild Journal Catalog...862gitlab # [ 7.010297] systemd[1]: Starting Record System Boot/Shutdown in UTMP...863gitlab # [ 7.030348] systemd[1]: Finished Record System Boot/Shutdown in UTMP.864gitlab # [ 7.042337] systemd[1]: Finished Rebuild Journal Catalog.865gitlab # [ 7.046407] systemd[1]: Starting Update is Completed...866gitlab # [ 7.062852] systemd[1]: Finished Update is Completed.867gitlab # [ 7.131160] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.868gitlab # [ 7.133441] systemd[1]: Finished Create SUID/SGID Wrappers.869gitlab # [ 7.346988] systemd[1]: Finished register-nix-paths.service.870gitlab # [ 7.348808] systemd[1]: Reached target System Initialization.871gitlab # [ 7.350482] systemd[1]: Started Discard unused filesystem blocks once a week.872gitlab # [ 7.352499] systemd[1]: Started logrotate.timer.873gitlab # [ 7.353762] systemd[1]: Started Daily Cleanup of Temporary Directories.874gitlab # [ 7.355574] systemd[1]: Reached target Timer Units.875gitlab # [ 7.356905] systemd[1]: Listening on D-Bus System Message Bus Socket.876gitlab # [ 7.358643] systemd[1]: Listening on Nix Daemon Socket.877gitlab # [ 7.360088] systemd[1]: Starting nixbot CI service socket...878gitlab # [ 7.361654] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.879gitlab # [ 7.366516] systemd[1]: Starting D-Bus System Message Bus...880gitlab # [ 7.368151] systemd[1]: Listening on nixbot CI service socket.881gitlab # [ 7.369857] systemd[1]: Reached target Socket Units.882gitlab # [ 7.443076] dbus-broker-launch[577]: Looking up NSS user entry for 'systemd-timesync'...883gitlab # [ 7.445650] dbus-broker-launch[577]: NSS returned no entry for 'systemd-timesync'884gitlab # [ 7.447966] dbus-broker-launch[577]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"885gitlab # [ 7.453203] systemd[1]: Started D-Bus System Message Bus.886gitlab # [ 7.473932] dbus-broker-launch[577]: Ready887gitlab # [ 7.476133] systemd[1]: Reached target Basic System.888gitlab # [ 7.479208] systemd[1]: Started backdoor.service.889gitlab # [ 7.482130] systemd[1]: Starting gitlab-config.service...890gitlab # [ 7.485341] systemd[1]: Starting Import lastlog data into lastlog2 database...891gitlab # [ 7.495222] systemd[1]: Starting Logrotate configuration check...892gitlab # [ 7.501187] systemd[1]: Starting Name Service Cache Daemon (nsncd)...893gitlab # [ 7.508184] systemd[1]: Starting Post-Boot Actions...894gitlab # [ 7.518914] systemd[1]: Started Reset console on configuration changes.895gitlab # connecting to host...896gitlab # [ 7.527243] systemd[1]: Starting resolvconf update...897gitlab # [ 7.528478] logrotate[582]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.898gitlab # [ 7.529440] logrotate[582]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf899gitlab # [ 7.533112] logrotate[582]: note: 'monthly' overrides previously specified 'weekly'900gitlab # [ 7.533599] logrotate[582]: Reading state from file: /var/lib/logrotate.status901gitlab # [ 7.534303] logrotate[582]: state file /var/lib/logrotate.status does not exist902gitlab # [ 7.537576] logrotate[582]: Allocating hash table for state file, size 64 entries903gitlab # [ 7.538439] logrotate[582]: Handling 4 logs904gitlab # [ 7.539640] logrotate[582]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed905gitlab # [ 7.540862] logrotate[582]: considering log /var/log/btmp906gitlab # [ 7.541354] logrotate[582]: Creating new state907gitlab # [ 7.541864] logrotate[582]: Now: 2026-11-12 09:19908gitlab # [ 7.542546] logrotate[582]: Last rotated at 2026-11-12 09:00909gitlab # [ 7.543017] logrotate[582]: log does not need rotating (log has already been rotated)910gitlab # [ 7.545764] logrotate[582]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed911gitlab # [ 7.546467] logrotate[582]: considering log /var/log/wtmp912gitlab # [ 7.546914] logrotate[582]: Creating new state913gitlab # [ 7.547677] logrotate[582]: Now: 2026-11-12 09:19914gitlab # [ 7.550419] logrotate[582]: Last rotated at 2026-11-12 09:00915gitlab # [ 7.552476] logrotate[582]: log does not need rotating (log has already been rotated)916gitlab # [ 7.552945] logrotate[582]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed917gitlab # [ 7.553608] logrotate[582]: switching euid from 0 to 165 and egid from 0 to 165 (pid 582)918gitlab # [ 7.554686] logrotate[582]: considering log /var/gitlab/state/log/*.log919gitlab # [ 7.557156] logrotate[582]: log /var/gitlab/state/log/*.log does not exist -- skipping920gitlab # [ 7.557867] logrotate[582]: Creating new state921gitlab # [ 7.558586] logrotate[582]: switching euid from 165 to 0 and egid from 165 to 0 (pid 582)922gitlab # [ 7.559090] logrotate[582]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed923gitlab # [ 7.559558] logrotate[582]: switching euid from 0 to 60 and egid from 0 to 60 (pid 582)924gitlab # [ 7.560050] logrotate[582]: considering log /var/log/nginx/*.log925gitlab # [ 7.560818] logrotate[582]: log /var/log/nginx/*.log does not exist -- skipping926gitlab # [ 7.562392] logrotate[582]: Creating new state927gitlab # [ 7.563120] logrotate[582]: not running postrotate script, since no logs were rotated928gitlab # [ 7.563364] logrotate[582]: switching euid from 60 to 0 and egid from 60 to 0 (pid 582)929gitlab # [ 7.564235] systemd[1]: Finished Logrotate configuration check.930gitlab # [ 7.565057] systemd[1]: Finished Post-Boot Actions.931gitlab # [ 7.565859] systemd[1]: Finished Import lastlog data into lastlog2 database.932gitlab: Guest shell says: b'Spawning backdoor root shell...\n'933gitlab: connected to guest root shell934gitlab # [ 7.585316] nsncd[583]: Nov 12 09:19:22.618 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"935gitlab: (connecting took 8.20 seconds)936gitlab # [ 7.585765] systemd[1]: Started Name Service Cache Daemon (nsncd).937gitlab: (finished: waiting for the VM to finish booting, in 8.20 seconds)938gitlab # [ 7.586456] systemd[1]: Reached target Host and Network Name Lookups.939gitlab # [ 7.586955] systemd[1]: Reached target User and Group Name Lookups.940gitlab # [ 7.588523] systemd[1]: Starting User Login Management...941gitlab # [ 7.624264] systemd[1]: Stopped target Host and Network Name Lookups.942gitlab # [ 7.625629] systemd[1]: Stopping Host and Network Name Lookups...943gitlab # [ 7.626458] systemd[1]: Stopped target User and Group Name Lookups.944gitlab # [ 7.627186] systemd[1]: Stopping User and Group Name Lookups...945gitlab # [ 7.627709] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...946gitlab # [ 7.629081] systemd[1]: nscd.service: Deactivated successfully.947gitlab # [ 7.630128] systemd[1]: Stopped Name Service Cache Daemon (nsncd).948gitlab # [ 7.630621] systemd[1]: nscd.service: Consumed 18ms CPU time over 127ms wall clock time, 7.4M memory peak, 5.5M read from disk.949gitlab # [ 7.634108] systemd[1]: Starting Name Service Cache Daemon (nsncd)...950gitlab # [ 7.636425] systemd-logind[634]: New seat seat0.951gitlab # [ 7.637622] systemd-logind[634]: Watching system buttons on /dev/input/event2 (Power Button)952gitlab # [ 7.638968] systemd-logind[634]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)953gitlab # [ 7.640407] systemd-logind[634]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)954gitlab # [ 7.641154] systemd[1]: Started User Login Management.955gitlab # [ 7.645370] systemd[1]: Starting linger-users.service...956gitlab # [ 7.655480] nsncd[673]: Nov 12 09:19:22.689 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"957gitlab # [ 7.656235] systemd[1]: Started Name Service Cache Daemon (nsncd).958gitlab # [ 7.656698] systemd[1]: Reached target Host and Network Name Lookups.959gitlab # [ 7.657294] systemd[1]: Reached target User and Group Name Lookups.960gitlab # [ 7.666653] systemd[1]: linger-users.service: Deactivated successfully.961gitlab # [ 7.667263] systemd[1]: Finished linger-users.service.962gitlab # [ 7.670722] systemd[1]: Finished resolvconf update.963gitlab # [ 7.671597] systemd[1]: Reached target Preparation for Network.964gitlab # [ 7.673857] systemd[1]: Starting DHCP Client...965gitlab # [ 7.675252] systemd[1]: Starting Address configuration of eth1...966gitlab # [ 7.677627] systemd[1]: Starting Extra networking commands....967gitlab # [ 7.706952] network-addresses-eth1-start[700]: adding address 192.168.1.1/24... done968gitlab # [ 7.710551] network-addresses-eth1-start[700]: adding address 2001:db8:1::1/64... done969gitlab # [ 7.714803] systemd[1]: Finished Address configuration of eth1.970gitlab # [ 7.743962] dhcpcd[709]: dhcpcd-10.3.2 starting971gitlab # [ 7.746602] dhcpcd[763]: dev: loaded udev972gitlab # [ 7.749200] systemd[1]: Finished Extra networking commands..973gitlab # [ 7.750405] systemd[1]: Reached target Network.974gitlab # [ 7.899056] 8021q: 802.1Q VLAN Support v1.8975gitlab # [ 7.899514] 8021q: adding VLAN 0 to HW filter on device eth1976gitlab # [ 7.753257] systemd[1]: Starting gitlab-workhorse.service...977gitlab # [ 7.756289] systemd[1]: Starting Nginx Web Server...978gitlab # [ 7.760322] systemd[1]: Starting PostgreSQL Server...979gitlab # [ 7.764366] systemd[1]: Starting Redis Server - redis-gitlab...980gitlab # [ 7.770385] systemd[1]: Starting Permit User Sessions...981gitlab # [ 7.774053] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.982gitlab # [ 7.798981] systemd[1]: Finished Permit User Sessions.983gitlab # [ 7.802377] systemd[1]: Started Getty on tty1.984gitlab # [ 7.803947] systemd[1]: Reached target Login Prompts.985gitlab # [ 7.987359] cfg80211: Loading compiled-in X.509 certificates for regulatory database986gitlab # [ 7.992915] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'987gitlab # [ 7.993753] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'988gitlab # [ 7.995303] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2989gitlab # [ 7.996369] cfg80211: failed to load regulatory.db990gitlab # [ 8.032655] 8021q: adding VLAN 0 to HW filter on device eth0991gitlab # [ 7.888204] dhcpcd[763]: eth0: waiting for carrier992gitlab # [ 7.889172] dhcpcd[763]: eth0: carrier acquired993gitlab # [ 7.897524] dhcpcd[763]: DUID 00:01:00:01:32:88:41:1a:52:54:00:12:34:56994gitlab # [ 7.898300] dhcpcd[763]: eth0: IAID 00:12:34:56995gitlab # [ 7.898871] dhcpcd[763]: eth0: adding address fe80::5054:ff:fe12:3456996gitlab # [ 7.909547] redis-server[789]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.997gitlab # [ 7.912163] redis-server[789]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo998gitlab # [ 7.913993] nginx-pre-start[793]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok999gitlab # [ 7.915952] nginx-pre-start[793]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful1000gitlab # [ 7.917617] systemd[1]: Started Nginx Web Server.1001gitlab # [ 7.918402] redis-server[789]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=789, just started1002gitlab # [ 7.919931] systemd[1]: Started Redis Server - redis-gitlab.1003gitlab # [ 7.920921] redis-server[789]: Configuration loaded1004gitlab # [ 7.921854] redis[789]: monotonic clock: POSIX clock_gettime1005gitlab # [ 7.922870] redis[789]: Running mode=standalone, port=0.1006gitlab # [ 7.923922] redis[789]: Server initialized1007gitlab # [ 7.924817] redis[789]: Ready to accept connections unix1008gitlab # [ 7.925994] redis[789]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1009gitlab # [ 7.928309] postgresql-pre-start[798]: The files belonging to this database system will be owned by user "postgres".1010gitlab # [ 7.930425] postgresql-pre-start[798]: This user must also own the server process.1011gitlab # [ 7.931809] postgresql-pre-start[798]: The database cluster will be initialized with locale "en_US.UTF-8".1012gitlab # [ 7.933315] postgresql-pre-start[798]: The default database encoding has accordingly been set to "UTF8".1013gitlab # [ 7.934759] postgresql-pre-start[798]: The default text search configuration will be set to "english".1014gitlab # [ 7.936269] postgresql-pre-start[798]: Data page checksums are enabled.1015gitlab # [ 7.937431] postgresql-pre-start[798]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok1016gitlab # [ 7.938801] postgresql-pre-start[798]: creating subdirectories ... ok1017gitlab # [ 7.939833] postgresql-pre-start[798]: selecting dynamic shared memory implementation ... posix1018gitlab # [ 7.961027] postgresql-pre-start[798]: selecting default "max_connections" ... 1001019gitlab # [ 8.012246] postgresql-pre-start[798]: selecting default "shared_buffers" ... 128MB1020gitlab # [ 8.292110] systemd[1]: Started gitlab-workhorse.service.1021gitlab # [ 8.463674] postgresql-pre-start[798]: selecting default time zone ... UTC1022gitlab # [ 8.466269] postgresql-pre-start[798]: creating configuration files ... ok1023gitlab # [ 8.494730] gitlab-workhorse[825]: time="2026-11-12T09:19:23Z" level=info msg=Starting build_time=19700101.000000 version=19.0.41024gitlab # [ 8.495319] gitlab-workhorse[825]: time="2026-11-12T09:19:23Z" level=info msg="Using redis/go-redis"1025gitlab # [ 8.495746] gitlab-workhorse[825]: time="2026-11-12T09:19:23Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1026gitlab # [ 8.637174] postgresql-pre-start[798]: running bootstrap script ... ok1027gitlab # [ 9.078509] postgresql-pre-start[798]: performing post-bootstrap initialization ... ok1028gitlab # [ 9.351795] postgresql-pre-start[798]: syncing data to disk ... ok1029gitlab # [ 9.353500] postgresql-pre-start[798]: initdb: warning: enabling "trust" authentication for local connections1030gitlab # [ 9.354905] postgresql-pre-start[798]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1031gitlab # [ 9.356887] postgresql-pre-start[798]: Success. You can now start the database server using:1032gitlab # [ 9.358487] postgresql-pre-start[798]: pg_ctl -D /var/lib/postgresql/18 -l logfile start1033gitlab # [ 9.496433] postgres[850]: [850] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1034gitlab # [ 9.499137] postgres[850]: [850] LOG: listening on IPv6 address "::1", port 54321035gitlab # [ 9.500235] postgres[850]: [850] LOG: listening on IPv4 address "127.0.0.1", port 54321036gitlab # [ 9.506220] postgres[850]: [850] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1037gitlab # [ 9.524631] postgres[859]: [859] LOG: database system was shut down at 2026-11-12 09:19:24 GMT1038gitlab # [ 9.534592] postgres[850]: [850] LOG: database system is ready to accept connections1039gitlab # [ 9.538128] systemd[1]: Started PostgreSQL Server.1040gitlab # [ 9.543932] systemd[1]: Starting PostgreSQL Setup Scripts...1041gitlab # [ 9.691562] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[866]: mkdir -p /var/gitlab/state/home/.ssh: OK1042gitlab # [ 9.693340] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[866]: chmod 700 /var/gitlab/state/home/.ssh: OK1043gitlab # [ 9.725985] dhcpcd[763]: eth0: soliciting a DHCP lease1044gitlab # [ 9.877572] NET: Registered PF_PACKET protocol family1045gitlab # [ 9.741565] dhcpcd[763]: eth0: offered 10.0.2.15 from 10.0.2.21046gitlab # [ 9.744885] dhcpcd[763]: eth0: probing address 10.0.2.15/241047gitlab # [ 9.745842] systemd[1]: Finished gitlab-config.service.1048gitlab # [ 9.750041] systemd[1]: Started gitaly.service.1049gitlab # [ 9.757549] postgresql-setup-start[875]: CREATE DATABASE1050gitlab # [ 9.778352] postgresql-setup-start[905]: CREATE ROLE1051gitlab # [ 9.789546] postgresql-setup-start[907]: ALTER DATABASE1052gitlab # [ 9.808615] postgresql-setup-start[912]: CREATE ROLE1053gitlab # [ 9.810703] systemd[1]: Finished PostgreSQL Setup Scripts.1054gitlab # [ 9.811760] systemd[1]: Reached target PostgreSQL.1055gitlab # [ 9.812533] systemd[1]: Starting gitlab-db-config.service...1056gitlab # [ 9.813689] systemd[1]: Starting gitlab-postgresql.service...1057gitlab # [ 9.900842] gitlab-postgresql-start[922]: CREATE DATABASE1058gitlab # [ 9.947715] gitlab-postgresql-start[933]: CREATE EXTENSION1059gitlab # [ 10.027903] gitlab-postgresql-start[936]: CREATE EXTENSION1060gitlab # [ 10.031104] systemd[1]: Finished gitlab-postgresql.service.1061gitlab # [ 10.214756] gitaly[899]: time="2026-11-12T09:19:25.248Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=8991062gitlab # [ 10.215670] gitaly[899]: time="2026-11-12T09:19:25.248Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=8991063gitlab # [ 10.218139] gitaly[899]: time="2026-11-12T09:19:25.248Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=8991064gitlab # [ 10.218214] gitaly[899]: time="2026-11-12T09:19:25.248Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=8991065gitlab # [ 10.219435] gitaly[899]: time="2026-11-12T09:19:25.248Z" level=info msg="Starting Gitaly" pid=899 version=1066gitlab # [ 10.407793] gitaly[899]: time="2026-11-12T09:19:25.441Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=192 pid=8991067gitlab # [ 10.408773] gitaly[899]: time="2026-11-12T09:19:25.443Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=8991068gitlab # [ 10.410067] gitaly[899]: time="2026-11-12T09:19:25.444Z" level=info msg="finished initializing command factory" duration_ms=0 pid=8991069gitlab # [ 10.410451] gitaly[899]: time="2026-11-12T09:19:25.444Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-899/git-exec-4083689159.d/git pid=8991070gitlab # [ 10.414731] gitaly[899]: time="2026-11-12T09:19:25.448Z" level=info msg="finished detecting git version" duration_ms=3 pid=8991071gitlab # [ 10.416130] gitaly[899]: time="2026-11-12T09:19:25.449Z" level=info msg="using Git version" pid=899 version=2.53.g1072gitlab # [ 10.416706] gitaly[899]: time="2026-11-12T09:19:25.450Z" level=info msg="clearing disk cache object folder" pid=899 storage=default1073gitlab # [ 10.417359] gitaly[899]: time="2026-11-12T09:19:25.451Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1286430678 pid=899 storage=default1074gitlab # [ 10.418184] gitaly[899]: time="2026-11-12T09:19:25.452Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1286430678 pid=899 storage=default1075gitlab # [ 10.418875] gitaly[899]: time="2026-11-12T09:19:25.452Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=8991076gitlab # [ 10.419466] gitaly[899]: time="2026-11-12T09:19:25.452Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=8991077gitlab # [ 10.419918] gitaly[899]: time="2026-11-12T09:19:25.452Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1286430678 clear_duration_ms=0 pid=899 storage=default1078gitlab # [ 10.454568] gitaly[899]: time="2026-11-12T09:19:25.488Z" level=info msg="counting repositories" pid=8991079gitlab # [ 10.455641] gitaly[899]: time="2026-11-12T09:19:25.489Z" level=info msg="starting to count repositories" pid=899 storage_path=/var/gitlab/state/repositories1080gitlab # [ 10.456238] gitaly[899]: time="2026-11-12T09:19:25.489Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=899 storage_path=/var/gitlab/state/repositories1081gitlab # [ 10.456687] gitaly[899]: time="2026-11-12T09:19:25.488Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=8991082gitlab # [ 10.457295] gitaly[899]: time="2026-11-12T09:19:25.488Z" level=info msg="finished tempdir cleaner walk" pid=899 storage=default time_ms=01083gitlab # [ 10.457726] gitaly[899]: time="2026-11-12T09:19:25.489Z" level=info msg="completed counting all repositories" counting_duration=0 pid=8991084gitlab # [ 10.458405] gitaly[899]: time="2026-11-12T09:19:25.490Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=899 schema=unix1085gitlab # [ 10.458876] gitaly[899]: time="2026-11-12T09:19:25.490Z" level=info msg="listening at address" address=/run/gitaly/gitaly-899/sock.d/intern pid=899 schema=unix1086gitlab # [ 10.459391] gitaly[899]: time="2026-11-12T09:19:25.491Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=8991087gitlab # [ 10.459904] gitaly[899]: time="2026-11-12T09:19:25.491Z" level=info msg="Started Gitaly" duration_ms=242 pid=8991088gitlab # [ 10.460591] gitaly[899]: time="2026-11-12T09:19:25.491Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=8991089gitlab # [ 10.472841] gitaly[899]: time="2026-11-12T09:19:25.506Z" level=info msg="maintenance: daily scheduled" pid=899 scheduled="2026-11-12 12:00:00 +0000 UTC"1090gitlab # [ 10.546267] dhcpcd[763]: eth0: soliciting an IPv6 router1091gitlab # [ 10.548024] dhcpcd[763]: eth0: Router Advertisement from fe80::21092gitlab # [ 10.549837] dhcpcd[763]: eth0: adding address fec0::5054:ff:fe12:3456/641093gitlab # [ 10.551647] dhcpcd[763]: eth0: adding route to fec0::/641094gitlab # [ 10.552743] dhcpcd[763]: eth0: adding default route via fe80::21095gitlab # [ 11.366583] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[917]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1096gitlab # [ 11.369306] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[917]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1097gitlab # [ 11.396698] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[917]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1098gitlab # [ 12.496653] gitaly[899]: time="2026-11-12T09:19:27.529Z" level=info msg="License database preloaded" duration_ms=2280 pid=8991099gitlab # [ 13.880490] dhcpcd[763]: eth0: leased 10.0.2.15 for 86400 seconds1100gitlab # [ 13.882425] dhcpcd[763]: eth0: adding route to 10.0.2.0/241101gitlab # [ 13.884241] dhcpcd[763]: eth0: adding default route via 10.0.2.21102gitlab # [ 14.010911] systemd[1]: Started DHCP Client.1103gitlab # [ 14.014227] systemd[1]: Reached target Network is Online.1104gitlab # [ 42.124391] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[917]: W, [2026-11-12T09:19:57.157458 #917] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1105gitlab # [ 44.386755] postgres[1317]: [1317] ERROR: relation "feature_gates" does not exist at character 5231106gitlab # [ 44.387965] postgres[1317]: [1317] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1107gitlab # [ 44.388655] postgres[1317]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1108gitlab # [ 44.389863] postgres[1317]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1109gitlab # [ 44.390849] postgres[1317]: attidentity AS identity,1110gitlab # [ 44.391863] postgres[1317]: attgenerated as attgenerated1111gitlab # [ 44.392984] postgres[1317]: FROM pg_attribute a1112gitlab # [ 44.394615] postgres[1317]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1113gitlab # [ 44.396134] postgres[1317]: LEFT JOIN pg_type t ON a.atttypid = t.oid1114gitlab # [ 44.397718] postgres[1317]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1115gitlab # [ 44.399192] postgres[1317]: WHERE a.attrelid = '"feature_gates"'::regclass1116gitlab # [ 44.399976] postgres[1317]: AND a.attnum > 0 AND NOT a.attisdropped1117gitlab # [ 44.400925] postgres[1317]: ORDER BY a.attnum1118gitlab # [ 44.402545] postgres[1317]:1119gitlab # [ 48.112204] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1359]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1120gitlab # [ 48.113462] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1359]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1121gitlab # [ 48.542484] gitlab-workhorse[825]: time="2026-11-12T09:20:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1122gitlab # [ 58.543694] gitlab-workhorse[825]: time="2026-11-12T09:20:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1123gitlab # [ 63.460256] postgres[1434]: [1434] ERROR: relation "table_without_models" does not exist1124gitlab # [ 63.462980] postgres[1434]: [1434] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1125gitlab # [ 68.552157] gitlab-workhorse[825]: time="2026-11-12T09:20:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1126gitlab # [ 76.407363] systemd[1]: Finished gitlab-db-config.service.1127gitlab # [ 76.411660] systemd[1]: Started gitlab-sidekiq.service.1128gitlab # [ 76.414301] systemd[1]: Starting gitlab.service...1129gitlab # [ 76.790669] bundle[1610]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1130gitlab # [ 76.795493] bundle[1610]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1131gitlab # [ 76.800106] bundle[1610]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1132gitlab # [ 78.562148] gitlab-workhorse[825]: time="2026-11-12T09:20:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1133gitlab # [ 79.659653] sidekiq-cluster[1609]: {"severity":"INFO","time":"2026-11-12T09:20:34.675Z","message":"Starting cluster with 1 processes"}1134gitlab # [ 79.705529] bundle[1610]: {"timestamp":"2026-11-12T09:20:34.718Z","pid":1610,"message":"* Enabling systemd notification integration"}1135gitlab # [ 79.709327] bundle[1610]: {"timestamp":"2026-11-12T09:20:34.738Z","pid":1610,"message":"Puma starting in cluster mode..."}1136gitlab # [ 79.712618] bundle[1610]: {"timestamp":"2026-11-12T09:20:34.738Z","pid":1610,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1137gitlab # [ 79.716333] bundle[1610]: {"timestamp":"2026-11-12T09:20:34.738Z","pid":1610,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1138gitlab # [ 79.720601] bundle[1610]: {"timestamp":"2026-11-12T09:20:34.739Z","pid":1610,"message":"* Min threads: 0"}1139gitlab # [ 79.723749] bundle[1610]: {"timestamp":"2026-11-12T09:20:34.739Z","pid":1610,"message":"* Max threads: 4"}1140gitlab # [ 79.726753] bundle[1610]: {"timestamp":"2026-11-12T09:20:34.739Z","pid":1610,"message":"* Environment: production"}1141gitlab # [ 79.729909] bundle[1610]: {"timestamp":"2026-11-12T09:20:34.739Z","pid":1610,"message":"* Master PID: 1610"}1142gitlab # [ 79.732943] bundle[1610]: {"timestamp":"2026-11-12T09:20:34.739Z","pid":1610,"message":"* Workers: 2"}1143gitlab # [ 79.736151] bundle[1610]: {"timestamp":"2026-11-12T09:20:34.739Z","pid":1610,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1144gitlab # [ 79.740301] bundle[1610]: {"timestamp":"2026-11-12T09:20:34.739Z","pid":1610,"message":"* Preloading application"}1145gitlab # [ 88.571641] gitlab-workhorse[825]: time="2026-11-12T09:20:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1146gitlab # [ 98.573105] gitlab-workhorse[825]: time="2026-11-12T09:20:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1147gitlab # [ 103.741741] bundle[1610]: W, [2026-11-12T09:20:58.775070 #1610] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1148gitlab # [ 106.251384] sidekiq-cluster[1640]: 2026-11-12T09:21:01.284Z pid=1640 tid=wjk INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1640, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>431177728}1149gitlab # [ 106.435198] sidekiq-cluster[1640]: W, [2026-11-12T09:21:01.468995 #1640] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1150gitlab # [ 106.477116] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:01.510Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1151gitlab # [ 108.538884] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.571Z","message":"GitLab reliable fetch activated!"}1152gitlab # [ 108.540196] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.572Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1153gitlab # [ 108.561491] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.595Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1154gitlab # [ 108.564200] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.598Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1155gitlab # [ 108.566131] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.600Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1156gitlab # [ 108.567916] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.602Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1157gitlab # [ 108.569876] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.604Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1158gitlab # [ 108.571801] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.605Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1159gitlab # [ 108.573502] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.607Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1160gitlab # [ 108.575053] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.609Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1161gitlab # [ 108.577504] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.611Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1162gitlab # [ 108.579809] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.613Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1163gitlab # [ 108.582066] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.616Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1164gitlab # [ 108.584202] gitlab-workhorse[825]: time="2026-11-12T09:21:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1165gitlab # [ 108.586262] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.620Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1166gitlab # [ 108.588542] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.622Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1167gitlab # [ 108.590840] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.624Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1168gitlab # [ 108.593877] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.627Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1169gitlab # [ 108.596242] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.630Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1170gitlab # [ 108.599281] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.633Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1171gitlab # [ 108.601939] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.635Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1172gitlab # [ 108.604823] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.638Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1173gitlab # [ 108.607901] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.642Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1174gitlab # [ 108.611180] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.645Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1175gitlab # [ 108.613965] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.648Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1176gitlab # [ 108.616379] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.650Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1177gitlab # [ 108.618762] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.652Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1178gitlab # [ 108.620781] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.654Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1179gitlab # [ 108.623360] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.657Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1180gitlab # [ 108.625804] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.659Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1181gitlab # [ 108.628278] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.662Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1182gitlab # [ 108.630787] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.664Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1183gitlab # [ 108.633324] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.667Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1184gitlab # [ 108.635663] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.669Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1185gitlab # [ 108.638267] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.672Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1186gitlab # [ 108.640674] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.674Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1187gitlab # [ 108.666279] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.700Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1188gitlab # [ 108.672335] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.706Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1189gitlab # [ 108.676450] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.710Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1190gitlab # [ 108.680939] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.715Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1191gitlab # [ 108.688228] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.722Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1192gitlab # [ 108.693710] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.727Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1193gitlab # [ 108.698685] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.732Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1194gitlab # [ 108.703701] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.737Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1195gitlab # [ 108.708196] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.742Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1196gitlab # [ 108.713191] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.747Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1197gitlab # [ 108.717630] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.751Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1198gitlab # [ 108.723219] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.757Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1199gitlab # [ 108.728315] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.762Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1200gitlab # [ 108.733083] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.767Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1201gitlab # [ 108.738110] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.772Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1202gitlab # [ 108.744100] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.778Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1203gitlab # [ 108.749097] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.783Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1204gitlab # [ 108.758051] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.791Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1205gitlab # [ 108.765760] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.799Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1206gitlab # [ 108.788760] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.822Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1207gitlab # [ 108.797225] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.831Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1208gitlab # [ 108.805101] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.839Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1209gitlab # [ 108.813060] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.847Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1210gitlab # [ 108.820199] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.854Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1211gitlab # [ 108.828793] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.862Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1212gitlab # [ 108.835365] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.869Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1213gitlab # [ 108.841570] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.875Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1214gitlab # [ 108.847624] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.881Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1215gitlab # [ 108.853982] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.888Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1216gitlab # [ 108.861683] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.895Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1217gitlab # [ 108.868645] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.902Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1218gitlab # [ 108.877102] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.911Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1219gitlab # [ 108.883916] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.917Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1220gitlab # [ 108.889357] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.923Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1221gitlab # [ 108.894702] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.928Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1222gitlab # [ 108.900119] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.934Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1223gitlab # [ 108.952824] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.986Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1224gitlab # [ 108.960122] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:03.994Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1225gitlab # [ 108.967459] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.001Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1226gitlab # [ 108.974430] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.008Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1227gitlab # [ 108.982319] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.016Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1228gitlab # [ 108.994394] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.028Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1229gitlab # [ 109.002882] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.036Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1230gitlab # [ 109.012907] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.046Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1231gitlab # [ 109.022840] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.056Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1232gitlab # [ 109.032637] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.066Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1233gitlab # [ 109.041208] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.075Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1234gitlab # [ 109.050242] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.084Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1235gitlab # [ 109.055541] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.089Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1236gitlab # [ 109.061807] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.095Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1237gitlab # [ 109.071298] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.105Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1238gitlab # [ 109.080647] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.114Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1239gitlab # [ 109.089349] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.123Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1240gitlab # [ 109.098093] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.132Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1241gitlab # [ 109.106592] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.140Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1242gitlab # [ 109.114606] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.148Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1243gitlab # [ 109.123203] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.157Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1244gitlab # [ 109.130882] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.164Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1245gitlab # [ 109.139284] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.173Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1246gitlab # [ 109.146665] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.180Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1247gitlab # [ 109.154683] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.188Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1248gitlab # [ 109.162823] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.196Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1249gitlab # [ 109.173171] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.207Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1250gitlab # [ 109.180801] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.214Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1251gitlab # [ 109.188961] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.222Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1252gitlab # [ 109.213965] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.247Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1253gitlab # [ 109.221725] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.255Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1254gitlab # [ 109.229668] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.263Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1255gitlab # [ 109.239497] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.273Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1256gitlab # [ 109.247119] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.281Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1257gitlab # [ 109.255699] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.289Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1258gitlab # [ 109.263519] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.297Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1259gitlab # [ 109.271314] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.305Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1260gitlab # [ 109.279514] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.313Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1261gitlab # [ 109.289991] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.324Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1262gitlab # [ 109.297708] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:04.331Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1263gitlab # [ 118.592976] gitlab-workhorse[825]: time="2026-11-12T09:21:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1264gitlab # [ 128.594669] gitlab-workhorse[825]: time="2026-11-12T09:21:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1265gitlab # [ 130.925082] postgres[2105]: [2105] ERROR: relation "table_without_models" does not exist1266gitlab # [ 130.925479] postgres[2105]: [2105] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1267gitlab # [ 131.243431] postgres[2117]: [2117] ERROR: relation "table_without_models" does not exist1268gitlab # [ 131.245186] postgres[2117]: [2117] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1269gitlab # [ 133.374221] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:28.407Z","message":"Booted Rails 7.2.3.1 application in production environment"}1270gitlab # [ 133.378054] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:28.411Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1271gitlab # [ 133.378679] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:28.411Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1272gitlab # [ 133.379235] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:28.411Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1273gitlab # [ 133.385678] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:28.419Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1274gitlab # [ 134.010323] bundle[1610]: {"timestamp":"2026-11-12T09:21:29.043Z","pid":1610,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1275gitlab # [ 134.011743] bundle[1610]: {"timestamp":"2026-11-12T09:21:29.043Z","pid":1610,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1276gitlab # [ 134.012674] bundle[1610]: {"timestamp":"2026-11-12T09:21:29.043Z","pid":1610,"message":"! #\u003cThread:0x00007d3ebf4ac600 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1277gitlab # [ 134.013643] bundle[1610]: {"timestamp":"2026-11-12T09:21:29.043Z","pid":1610,"message":"! #\u003cThread:0x00007d3ebf75c328 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1278gitlab # [ 134.014472] bundle[1610]: {"timestamp":"2026-11-12T09:21:29.043Z","pid":1610,"message":"Use Ctrl-C to stop"}1279gitlab # [ 134.015161] bundle[1610]: {"timestamp":"2026-11-12T09:21:29.044Z","pid":1610,"message":"Sending status to systemd every 1.0 sec"}1280gitlab # [ 134.416165] bundle[1610]: {"timestamp":"2026-11-12T09:21:29.449Z","pid":1610,"message":"- Worker 0 (PID: 2174) booted in 0.34s, phase: 0"}1281gitlab # [ 134.487268] bundle[1610]: {"timestamp":"2026-11-12T09:21:29.521Z","pid":1610,"message":"- Worker 1 (PID: 2178) booted in 0.39s, phase: 0"}1282gitlab # [ 134.495769] systemd[1]: Started gitlab.service.1283gitlab # [ 134.496571] systemd[1]: Reached target Common target for all GitLab services..1284gitlab # [ 134.499385] systemd[1]: Starting setup-gitlab.service...1285gitlab # [ 134.529840] setup-gitlab-start[2202]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1286gitlab # [ 135.241628] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:30.274Z","message":"Cleaning working queues"}1287gitlab # [ 135.247126] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:30.280Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c7f0c43fde57c9ba3aca9cef","created_at":"2026-11-12T09:20:30.912Z","trace_propagation_headers":{"sentry-trace":"6c76ef1cf6924a7b80eef1aa12f95b48-6e0a792e2cd742e2","baggage":"sentry-trace_id=6c76ef1cf6924a7b80eef1aa12f95b48,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"82c335dcaece0e5d2eee4a990a7b93ee","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-12T09:20:30.920Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c7f0c43fde57c9ba3aca9cef: start","job_status":"start","queue_duration_s":59.360451,"scheduling_latency_s":59.360531}1288gitlab # [ 135.452025] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:21:30.485Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c7f0c43fde57c9ba3aca9cef","created_at":"2026-11-12T09:20:30.912Z","trace_propagation_headers":{"sentry-trace":"6c76ef1cf6924a7b80eef1aa12f95b48-6e0a792e2cd742e2","baggage":"sentry-trace_id=6c76ef1cf6924a7b80eef1aa12f95b48,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"82c335dcaece0e5d2eee4a990a7b93ee","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-12T09:20:30.920Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c7f0c43fde57c9ba3aca9cef: done: 0.205216 sec","job_status":"done","queue_duration_s":59.360451,"scheduling_latency_s":59.360531,"redis_calls":26,"redis_duration_s":0.0076040000000000005,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.003995,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00219,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001419,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.07823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.088239707,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.205216,"completed_at":"2026-11-12T09:21:30.485Z","load_balancing_strategy":"primary","db_duration_s":0.049654,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1289gitlab # [ 136.959388] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:21:31 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 23911290gitlab # [ 136.966287] setup-gitlab-start[2202]: + TOKEN=nixbot-test-token-of-20-chars1291gitlab # [ 136.967457] setup-gitlab-start[2202]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1292gitlab # [ 136.990226] runuser[2227]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1293gitlab # [ 137.364464] setup-gitlab-start[2232]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1294gitlab # [ 137.366105] setup-gitlab-start[2232]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1295gitlab # [ 137.367248] setup-gitlab-start[2232]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1296gitlab # [ 137.957401] setup-gitlab-start[2232]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1297gitlab # [ 137.959117] setup-gitlab-start[2232]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1298gitlab # [ 137.960431] setup-gitlab-start[2232]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1299gitlab # [ 163.508324] setup-gitlab-start[2232]: W, [2026-11-12T09:21:58.541664 #2232] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1300gitlab # [ 166.653505] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:01.686Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"66c803b0305001544a035e8c","created_at":"2026-11-12T09:22:01.584Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b052eabc9a0a0549120c63097d203d09","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:01.683Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-66c803b0305001544a035e8c: start","job_status":"start","queue_duration_s":0.003037,"scheduling_latency_s":0.003142}1301gitlab # [ 166.799132] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:01.832Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"66c803b0305001544a035e8c","created_at":"2026-11-12T09:22:01.584Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b052eabc9a0a0549120c63097d203d09","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:01.683Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-66c803b0305001544a035e8c: done: 0.146112 sec","job_status":"done","queue_duration_s":0.003037,"scheduling_latency_s":0.003142,"redis_calls":12,"redis_duration_s":0.010029,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.001231,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000945,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005749,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002104,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001030298,"duration_s":0.146112,"completed_at":"2026-11-12T09:22:01.832Z","load_balancing_strategy":"primary","db_duration_s":0.020058,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1302gitlab # [ 166.866786] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:01.900Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f0c8b441e3232dc1a415ac43","created_at":"2026-11-12T09:22:01.705Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"743e70eb73cbcc4312f8e619bba037f6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:01.722Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f0c8b441e3232dc1a415ac43: start","job_status":"start","queue_duration_s":0.17847,"scheduling_latency_s":0.178508}1303gitlab # [ 166.931280] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:01.965Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1304gitlab # [ 166.947525] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:01.981Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f0c8b441e3232dc1a415ac43","created_at":"2026-11-12T09:22:01.705Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"743e70eb73cbcc4312f8e619bba037f6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:01.722Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f0c8b441e3232dc1a415ac43: done: 0.080717 sec","job_status":"done","queue_duration_s":0.17847,"scheduling_latency_s":0.178508,"redis_calls":10,"redis_duration_s":0.011899999999999999,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001108,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002727,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001565283,"duration_s":0.080717,"completed_at":"2026-11-12T09:22:01.981Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007177,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1305gitlab # [ 167.000516] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.034Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6454f47a159d1c894bedcad7","created_at":"2026-11-12T09:22:01.776Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"98ed0e5269d8ad8bebb5e34b7efb705b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:01.786Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6454f47a159d1c894bedcad7: start","job_status":"start","queue_duration_s":0.247773,"scheduling_latency_s":0.247826}1306gitlab # [ 167.071227] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.105Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6454f47a159d1c894bedcad7","created_at":"2026-11-12T09:22:01.776Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"98ed0e5269d8ad8bebb5e34b7efb705b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:01.786Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6454f47a159d1c894bedcad7: done: 0.071191 sec","job_status":"done","queue_duration_s":0.247773,"scheduling_latency_s":0.247826,"redis_calls":19,"redis_duration_s":0.014634,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001945,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000446,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.008985,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.003258,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000852064,"duration_s":0.071191,"completed_at":"2026-11-12T09:22:02.104Z","load_balancing_strategy":"primary","db_duration_s":0.004906,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1307gitlab # [ 167.115803] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.149Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"7d0ec1e2da421ef0e7be214d","created_at":"2026-11-12T09:22:01.800Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"023e3f68a20025fef0e17fe3e873bec1","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:01.805Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-7d0ec1e2da421ef0e7be214d: start","job_status":"start","queue_duration_s":0.344,"scheduling_latency_s":0.344046}1308gitlab # [ 167.222964] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.256Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"7d0ec1e2da421ef0e7be214d","created_at":"2026-11-12T09:22:01.800Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"023e3f68a20025fef0e17fe3e873bec1","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:01.805Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-7d0ec1e2da421ef0e7be214d: done: 0.107263 sec","job_status":"done","queue_duration_s":0.344,"scheduling_latency_s":0.344046,"redis_calls":16,"redis_duration_s":0.016293,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.008064,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005715,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002514,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036694,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00142532,"duration_s":0.107263,"completed_at":"2026-11-12T09:22:02.256Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02673,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 167.248739] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.263Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"42595d33433b719d26cc64cf","created_at":"2026-11-12T09:22:01.814Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e67dfcc7d7e3e24a75ba757c9db3d14c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:01.833Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-42595d33433b719d26cc64cf: start","job_status":"start","queue_duration_s":0.429499,"scheduling_latency_s":0.42958}1310gitlab # [ 167.355339] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.388Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"42595d33433b719d26cc64cf","created_at":"2026-11-12T09:22:01.814Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e67dfcc7d7e3e24a75ba757c9db3d14c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:01.833Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-42595d33433b719d26cc64cf: done: 0.125633 sec","job_status":"done","queue_duration_s":0.429499,"scheduling_latency_s":0.42958,"redis_calls":12,"redis_duration_s":0.048792,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004042,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.036947,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.007803,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001388165,"exclusive_lock_hold_duration_s":0.038498188999999995,"duration_s":0.125633,"completed_at":"2026-11-12T09:22:02.388Z","load_balancing_strategy":"primary","db_duration_s":0.029652,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1311gitlab # [ 167.416366] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.449Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"956a966bbe0d389e3363a8e0","created_at":"2026-11-12T09:22:01.930Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9cc789102c1830907dee38a984eb58cf","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:01.947Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-956a966bbe0d389e3363a8e0: start","job_status":"start","queue_duration_s":0.502075,"scheduling_latency_s":0.502145}1312gitlab # [ 167.802950] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"956a966bbe0d389e3363a8e0","created_at":"2026-11-12T09:22:01.930Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9cc789102c1830907dee38a984eb58cf","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:01.947Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-956a966bbe0d389e3363a8e0: done: 0.386493 sec","job_status":"done","queue_duration_s":0.502075,"scheduling_latency_s":0.502145,"redis_calls":565,"redis_duration_s":0.194438,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002068,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004253,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.188117,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.200348,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000372952,"duration_s":0.386493,"completed_at":"2026-11-12T09:22:02.835Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001947,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1313gitlab # [ 167.806416] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.837Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"637a63b7cc79635c9467a117","created_at":"2026-11-12T09:22:01.964Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1227d1d71a1d2be4a0d93740933dd434","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:01.979Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-637a63b7cc79635c9467a117: start","job_status":"start","queue_duration_s":0.857846,"scheduling_latency_s":0.857863}1314gitlab # [ 167.872299] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.858Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"637a63b7cc79635c9467a117","created_at":"2026-11-12T09:22:01.964Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1227d1d71a1d2be4a0d93740933dd434","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:01.979Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-637a63b7cc79635c9467a117: done: 0.021083 sec","job_status":"done","queue_duration_s":0.857846,"scheduling_latency_s":0.857863,"redis_calls":10,"redis_duration_s":0.001422,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000289,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000217346,"duration_s":0.021083,"completed_at":"2026-11-12T09:22:02.858Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003449,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1315gitlab # [ 167.873850] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.859Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"50c87fc0a607c112a069d916","created_at":"2026-11-12T09:22:02.035Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a13ae5264b8aec740e5c26900e2fdb2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.061Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-50c87fc0a607c112a069d916: start","job_status":"start","queue_duration_s":0.797829,"scheduling_latency_s":0.797843}1316gitlab # [ 167.886295] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.873Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"50c87fc0a607c112a069d916","created_at":"2026-11-12T09:22:02.035Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a13ae5264b8aec740e5c26900e2fdb2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.061Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-50c87fc0a607c112a069d916: done: 0.014314 sec","job_status":"done","queue_duration_s":0.797829,"scheduling_latency_s":0.797843,"redis_calls":10,"redis_duration_s":0.001322,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000274,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000787,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00020254,"duration_s":0.014314,"completed_at":"2026-11-12T09:22:02.873Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001309,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1317gitlab # [ 167.946025] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.874Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"00fa90833342a12856b58ada","created_at":"2026-11-12T09:22:02.074Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c9d55b9d663e043ab3b627b08f71f29","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-12T09:22:02.089Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-00fa90833342a12856b58ada: start","job_status":"start","queue_duration_s":0.784768,"scheduling_latency_s":0.784783}1318gitlab # [ 167.946771] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.882Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1319gitlab # [ 167.947466] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.886Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"00fa90833342a12856b58ada","created_at":"2026-11-12T09:22:02.074Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c9d55b9d663e043ab3b627b08f71f29","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-12T09:22:02.089Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-00fa90833342a12856b58ada: done: 0.012127 sec","job_status":"done","queue_duration_s":0.784768,"scheduling_latency_s":0.784783,"redis_calls":10,"redis_duration_s":0.002171,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000383,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001239,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000549,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000265397,"duration_s":0.012127,"completed_at":"2026-11-12T09:22:02.886Z","load_balancing_strategy":"primary","db_duration_s":0.000236,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 167.990573] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.887Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3bc56db5ecbfe8c33484a9b6","created_at":"2026-11-12T09:22:02.075Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-02c8aac56dfb4ffd","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"98ed0e5269d8ad8bebb5e34b7efb705b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.096Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3bc56db5ecbfe8c33484a9b6: start","job_status":"start","queue_duration_s":0.791229,"scheduling_latency_s":0.791242}1321gitlab # [ 167.991190] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.904Z","message":"No existing merge request to be cleaned up."}1322gitlab # [ 167.991672] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.912Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3bc56db5ecbfe8c33484a9b6","created_at":"2026-11-12T09:22:02.075Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-02c8aac56dfb4ffd","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"98ed0e5269d8ad8bebb5e34b7efb705b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.096Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3bc56db5ecbfe8c33484a9b6: done: 0.024624 sec","job_status":"done","queue_duration_s":0.791229,"scheduling_latency_s":0.791242,"redis_calls":13,"redis_duration_s":0.002819,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000421,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000834,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001564,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016794,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000213435,"duration_s":0.024624,"completed_at":"2026-11-12T09:22:02.912Z","load_balancing_strategy":"primary","db_duration_s":0.002625,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1323gitlab # [ 168.035122] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.913Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"108ff130571003a6b7702887","created_at":"2026-11-12T09:22:02.075Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-02c8aac56dfb4ffd","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"98ed0e5269d8ad8bebb5e34b7efb705b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.096Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-108ff130571003a6b7702887: start","job_status":"start","queue_duration_s":0.81678,"scheduling_latency_s":0.816794}1324gitlab # [ 168.035829] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.920Z","message":"No existing merge request to be cleaned up."}1325gitlab # [ 168.036681] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.924Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"108ff130571003a6b7702887","created_at":"2026-11-12T09:22:02.075Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-02c8aac56dfb4ffd","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"98ed0e5269d8ad8bebb5e34b7efb705b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.096Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-108ff130571003a6b7702887: done: 0.011455 sec","job_status":"done","queue_duration_s":0.81678,"scheduling_latency_s":0.816794,"redis_calls":10,"redis_duration_s":0.001414,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000887,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007556,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011455,"completed_at":"2026-11-12T09:22:02.924Z","load_balancing_strategy":"primary","db_duration_s":0.001173,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1326gitlab # [ 168.104842] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.925Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"59e0173f08c604998b262f5b","created_at":"2026-11-12T09:22:02.075Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-02c8aac56dfb4ffd","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"98ed0e5269d8ad8bebb5e34b7efb705b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.096Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-59e0173f08c604998b262f5b: start","job_status":"start","queue_duration_s":0.829094,"scheduling_latency_s":0.829107}1327gitlab # [ 168.105666] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.932Z","message":"No existing merge request to be cleaned up."}1328gitlab # [ 168.106214] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.936Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"59e0173f08c604998b262f5b","created_at":"2026-11-12T09:22:02.075Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-02c8aac56dfb4ffd","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"98ed0e5269d8ad8bebb5e34b7efb705b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.096Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-59e0173f08c604998b262f5b: done: 0.011096 sec","job_status":"done","queue_duration_s":0.829094,"scheduling_latency_s":0.829107,"redis_calls":10,"redis_duration_s":0.001388,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000845,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007359,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011096,"completed_at":"2026-11-12T09:22:02.936Z","load_balancing_strategy":"primary","db_duration_s":0.000962,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 168.148978] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.937Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8f4c12b3d8d63b63923ef96a","created_at":"2026-11-12T09:22:02.075Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-02c8aac56dfb4ffd","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"98ed0e5269d8ad8bebb5e34b7efb705b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.096Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8f4c12b3d8d63b63923ef96a: start","job_status":"start","queue_duration_s":0.841072,"scheduling_latency_s":0.841085}1330gitlab # [ 168.149638] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.944Z","message":"No existing merge request to be cleaned up."}1331gitlab # [ 168.150153] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.948Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8f4c12b3d8d63b63923ef96a","created_at":"2026-11-12T09:22:02.075Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-02c8aac56dfb4ffd","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"98ed0e5269d8ad8bebb5e34b7efb705b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.096Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8f4c12b3d8d63b63923ef96a: done: 0.011005 sec","job_status":"done","queue_duration_s":0.841072,"scheduling_latency_s":0.841085,"redis_calls":10,"redis_duration_s":0.00136,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00085,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011005,"completed_at":"2026-11-12T09:22:02.948Z","load_balancing_strategy":"primary","db_duration_s":0.000995,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1332gitlab # [ 168.195022] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:02.949Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"12248862c8ecd9323c3c6898","created_at":"2026-11-12T09:22:02.097Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c51dd7f0dac604f5d7c4ae54624a4f88","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.146Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-12248862c8ecd9323c3c6898: start","job_status":"start","queue_duration_s":0.803396,"scheduling_latency_s":0.803409}1333gitlab # [ 168.252103] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:03.285Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"12248862c8ecd9323c3c6898","created_at":"2026-11-12T09:22:02.097Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c51dd7f0dac604f5d7c4ae54624a4f88","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.146Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-12248862c8ecd9323c3c6898: done: 0.335895 sec","job_status":"done","queue_duration_s":0.803396,"scheduling_latency_s":0.803409,"redis_calls":14,"redis_duration_s":0.00231,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000551,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000959,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0008,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000323784,"exclusive_lock_hold_duration_s":0.316961514,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.335895,"completed_at":"2026-11-12T09:22:03.285Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.29815,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 168.292612] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:03.326Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"92294b59c08b222223fa69b4","created_at":"2026-11-12T09:22:02.164Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"030869ec897c5f8a55ff37875dd858b1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.181Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-92294b59c08b222223fa69b4: start","job_status":"start","queue_duration_s":1.144395,"scheduling_latency_s":1.144506}1335gitlab # [ 168.321117] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:03.354Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1336gitlab # [ 168.333625] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:03.367Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"92294b59c08b222223fa69b4","created_at":"2026-11-12T09:22:02.164Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"030869ec897c5f8a55ff37875dd858b1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.181Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-92294b59c08b222223fa69b4: done: 0.040901 sec","job_status":"done","queue_duration_s":1.144395,"scheduling_latency_s":1.144506,"redis_calls":10,"redis_duration_s":0.004353,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000483,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003306,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000564,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000461512,"duration_s":0.040901,"completed_at":"2026-11-12T09:22:03.366Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1337gitlab # [ 168.373134] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:03.406Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ef74cd57f13f1248258df4a8","created_at":"2026-11-12T09:22:02.197Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.207Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ef74cd57f13f1248258df4a8: start","job_status":"start","queue_duration_s":1.199131,"scheduling_latency_s":1.199192}1338gitlab # [ 168.526818] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:03.560Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ef74cd57f13f1248258df4a8","created_at":"2026-11-12T09:22:02.197Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.207Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ef74cd57f13f1248258df4a8: done: 0.154122 sec","job_status":"done","queue_duration_s":1.199131,"scheduling_latency_s":1.199192,"redis_calls":71,"redis_duration_s":0.021436999999999998,"redis_read_bytes":120,"redis_write_bytes":42574,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002874,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.00107,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37385,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.016512,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.126259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000486933,"duration_s":0.154122,"completed_at":"2026-11-12T09:22:03.560Z","load_balancing_strategy":"primary","db_duration_s":0.001883,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1339gitlab # [ 168.603457] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:03.637Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"04fe3d3f5de4db190e798338","created_at":"2026-11-12T09:22:02.214Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3582489107055d7a2c7c32e58553c2c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.224Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-04fe3d3f5de4db190e798338: start","job_status":"start","queue_duration_s":1.412451,"scheduling_latency_s":1.412506}1340gitlab # [ 168.647538] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:03.680Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"04fe3d3f5de4db190e798338","created_at":"2026-11-12T09:22:02.214Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3582489107055d7a2c7c32e58553c2c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.224Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-04fe3d3f5de4db190e798338: done: 0.043951 sec","job_status":"done","queue_duration_s":1.412451,"scheduling_latency_s":1.412506,"redis_calls":12,"redis_duration_s":0.002683,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000889,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001307,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000487,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000472686,"duration_s":0.043951,"completed_at":"2026-11-12T09:22:03.680Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00095,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1341gitlab # [ 168.691858] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:03.725Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"c9f5c907856f51d9b77ea8b6","created_at":"2026-11-12T09:22:02.236Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"809467d5dbf6e1d730de116845764374","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.261Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-c9f5c907856f51d9b77ea8b6: start","job_status":"start","queue_duration_s":1.464223,"scheduling_latency_s":1.464268}1342gitlab # [ 168.750691] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:03.784Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"c9f5c907856f51d9b77ea8b6","created_at":"2026-11-12T09:22:02.236Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"809467d5dbf6e1d730de116845764374","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.261Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-c9f5c907856f51d9b77ea8b6: done: 0.058772 sec","job_status":"done","queue_duration_s":1.464223,"scheduling_latency_s":1.464268,"redis_calls":13,"redis_duration_s":0.00392,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000511,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002078,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001331,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.014984866,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.058772,"completed_at":"2026-11-12T09:22:03.784Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003244,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1343gitlab # [ 168.800875] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:03.834Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c4b5ee28f3ef64032b851d56","created_at":"2026-11-12T09:22:02.278Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4fcba8a0f5965cfa60fd5f2c0299dac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-12T09:22:02.323Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c4b5ee28f3ef64032b851d56: start","job_status":"start","queue_duration_s":1.51082,"scheduling_latency_s":1.510875}1344gitlab # [ 168.847924] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:03.880Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c4b5ee28f3ef64032b851d56","created_at":"2026-11-12T09:22:02.278Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4fcba8a0f5965cfa60fd5f2c0299dac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-12T09:22:02.323Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c4b5ee28f3ef64032b851d56: done: 0.046544 sec","job_status":"done","queue_duration_s":1.51082,"scheduling_latency_s":1.510875,"redis_calls":10,"redis_duration_s":0.003258,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00065,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001964,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000644,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000502299,"duration_s":0.046544,"completed_at":"2026-11-12T09:22:03.880Z","load_balancing_strategy":"primary","db_duration_s":0.005345,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1345gitlab # [ 168.886915] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:03.920Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1774140bbdcfeb01d4fc9a82","created_at":"2026-11-12T09:22:02.357Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59bda131dc8270727d63831ba3c1e868","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.372Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1774140bbdcfeb01d4fc9a82: start","job_status":"start","queue_duration_s":1.548398,"scheduling_latency_s":1.548458}1346gitlab # [ 168.946427] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:03.979Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1774140bbdcfeb01d4fc9a82","created_at":"2026-11-12T09:22:02.357Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59bda131dc8270727d63831ba3c1e868","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.372Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1774140bbdcfeb01d4fc9a82: done: 0.05956 sec","job_status":"done","queue_duration_s":1.548398,"scheduling_latency_s":1.548458,"redis_calls":10,"redis_duration_s":0.002429,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001479,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000535,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000432178,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.05956,"completed_at":"2026-11-12T09:22:03.979Z","load_balancing_strategy":"primary","db_duration_s":0.015489,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 168.995761] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.029Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"30f52ef222b14d4bbf5db0b1","created_at":"2026-11-12T09:22:02.442Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d6ac19887731a42c641a3c945800d8e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.454Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-30f52ef222b14d4bbf5db0b1: start","job_status":"start","queue_duration_s":1.574451,"scheduling_latency_s":1.574498}1348gitlab # [ 169.015783] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.049Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1349gitlab # [ 169.026830] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.060Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"30f52ef222b14d4bbf5db0b1","created_at":"2026-11-12T09:22:02.442Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d6ac19887731a42c641a3c945800d8e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.454Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-30f52ef222b14d4bbf5db0b1: done: 0.031123 sec","job_status":"done","queue_duration_s":1.574451,"scheduling_latency_s":1.574498,"redis_calls":10,"redis_duration_s":0.003105,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000678,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001797,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00063,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000571023,"duration_s":0.031123,"completed_at":"2026-11-12T09:22:04.060Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000752,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 169.066921] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.100Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"02212b249ff6fe3f50286201","created_at":"2026-11-12T09:22:02.475Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b57ed1054af1dc13a2067f374156fd60","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.489Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-02212b249ff6fe3f50286201: start","job_status":"start","queue_duration_s":1.610953,"scheduling_latency_s":1.610999}1351gitlab # [ 169.096388] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.130Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1352gitlab # [ 169.109866] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.143Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"02212b249ff6fe3f50286201","created_at":"2026-11-12T09:22:02.475Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b57ed1054af1dc13a2067f374156fd60","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.489Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-02212b249ff6fe3f50286201: done: 0.043007 sec","job_status":"done","queue_duration_s":1.610953,"scheduling_latency_s":1.610999,"redis_calls":10,"redis_duration_s":0.004534,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003623,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000462,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000437485,"duration_s":0.043007,"completed_at":"2026-11-12T09:22:04.143Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000958,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353gitlab # [ 169.150858] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.184Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"fb398d09bd3d044149de7867","created_at":"2026-11-12T09:22:02.496Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49e4ed2577aaa9426da8bdba21cc2993","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.504Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-fb398d09bd3d044149de7867: start","job_status":"start","queue_duration_s":1.680034,"scheduling_latency_s":1.68009}1354gitlab # [ 169.182145] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.215Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"fb398d09bd3d044149de7867","created_at":"2026-11-12T09:22:02.496Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49e4ed2577aaa9426da8bdba21cc2993","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.504Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-fb398d09bd3d044149de7867: done: 0.031432 sec","job_status":"done","queue_duration_s":1.680034,"scheduling_latency_s":1.68009,"redis_calls":10,"redis_duration_s":0.003072,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000619,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001847,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000606,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047967,"duration_s":0.031432,"completed_at":"2026-11-12T09:22:04.215Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000826,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 169.224171] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.257Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9dfcf276f7a08cf0f58d542b","created_at":"2026-11-12T09:22:02.522Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd6b318921cdd21b89419435fba1a0ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.529Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9dfcf276f7a08cf0f58d542b: start","job_status":"start","queue_duration_s":1.727192,"scheduling_latency_s":1.727236}1356gitlab # [ 169.431221] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.464Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9dfcf276f7a08cf0f58d542b","created_at":"2026-11-12T09:22:02.522Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd6b318921cdd21b89419435fba1a0ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.529Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9dfcf276f7a08cf0f58d542b: done: 0.20786 sec","job_status":"done","queue_duration_s":1.727192,"scheduling_latency_s":1.727236,"redis_calls":14,"redis_duration_s":0.004574,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001468,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001758,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001348,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.149914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000773283,"exclusive_lock_hold_duration_s":0.173734092,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.20786,"completed_at":"2026-11-12T09:22:04.464Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.056049,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357gitlab # [ 169.505410] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.538Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2e2d65bc39c1934a5bf5eba7","created_at":"2026-11-12T09:22:02.544Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2c3e2e234ee0cef2eba8349aa9e6b8d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.551Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2e2d65bc39c1934a5bf5eba7: start","job_status":"start","queue_duration_s":1.987258,"scheduling_latency_s":1.987312}1358gitlab # [ 169.578606] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.611Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2e2d65bc39c1934a5bf5eba7","created_at":"2026-11-12T09:22:02.544Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c2c3e2e234ee0cef2eba8349aa9e6b8d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.551Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2e2d65bc39c1934a5bf5eba7: done: 0.073267 sec","job_status":"done","queue_duration_s":1.987258,"scheduling_latency_s":1.987312,"redis_calls":12,"redis_duration_s":0.00335,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000439,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001561,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00135,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00107807,"exclusive_lock_hold_duration_s":0.017192409,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.073267,"completed_at":"2026-11-12T09:22:04.611Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017815,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359gitlab # [ 169.643330] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.676Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5ee721b08fd3ee023403c43b","created_at":"2026-11-12T09:22:02.561Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"566f878b20bfce96ebee73e83770fc92","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.566Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5ee721b08fd3ee023403c43b: start","job_status":"start","queue_duration_s":2.110368,"scheduling_latency_s":2.11042}1360gitlab # [ 169.681507] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.715Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5ee721b08fd3ee023403c43b","created_at":"2026-11-12T09:22:02.561Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"566f878b20bfce96ebee73e83770fc92","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.566Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5ee721b08fd3ee023403c43b: done: 0.038267 sec","job_status":"done","queue_duration_s":2.110368,"scheduling_latency_s":2.11042,"redis_calls":12,"redis_duration_s":0.003691,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000613,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001683,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001395,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000885308,"exclusive_lock_hold_duration_s":0.006977702,"duration_s":0.038267,"completed_at":"2026-11-12T09:22:04.714Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003395,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361gitlab # [ 169.721617] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.755Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"c9ab620737e3cf6e9bf4fb16","created_at":"2026-11-12T09:22:02.570Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37a29f94f6f887ccaf36a2b05643d1b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-12T09:22:02.579Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-c9ab620737e3cf6e9bf4fb16: start","job_status":"start","queue_duration_s":2.175706,"scheduling_latency_s":2.175747}1362gitlab # [ 169.751600] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.785Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1363gitlab # [ 169.762857] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.796Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"c9ab620737e3cf6e9bf4fb16","created_at":"2026-11-12T09:22:02.570Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37a29f94f6f887ccaf36a2b05643d1b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-12T09:22:02.579Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-c9ab620737e3cf6e9bf4fb16: done: 0.041097 sec","job_status":"done","queue_duration_s":2.175706,"scheduling_latency_s":2.175747,"redis_calls":10,"redis_duration_s":0.003801,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00072,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002406,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000675,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000564877,"duration_s":0.041097,"completed_at":"2026-11-12T09:22:04.796Z","load_balancing_strategy":"primary","db_duration_s":0.000833,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 169.802731] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.836Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"2550c36a456af187ff194eb8","created_at":"2026-11-12T09:22:02.589Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.598Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-2550c36a456af187ff194eb8: start","job_status":"start","queue_duration_s":2.237452,"scheduling_latency_s":2.237505}1365gitlab # [ 169.875485] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.908Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"2550c36a456af187ff194eb8","created_at":"2026-11-12T09:22:02.589Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.598Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-2550c36a456af187ff194eb8: done: 0.072249 sec","job_status":"done","queue_duration_s":2.237452,"scheduling_latency_s":2.237505,"redis_calls":28,"redis_duration_s":0.008616,"redis_read_bytes":36,"redis_write_bytes":10593,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001342,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.00063,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8305,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.00457,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002074,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051255,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000466819,"duration_s":0.072249,"completed_at":"2026-11-12T09:22:04.908Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005485,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 169.934920] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:04.968Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"6bc98aeb1521418d334552a3","created_at":"2026-11-12T09:22:02.613Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87927d913aab579c2cb3b32f9a661e26","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.620Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-6bc98aeb1521418d334552a3: start","job_status":"start","queue_duration_s":2.347949,"scheduling_latency_s":2.347997}1367gitlab # [ 169.979782] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.013Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"6bc98aeb1521418d334552a3","created_at":"2026-11-12T09:22:02.613Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-615f1d946c4746ad","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87927d913aab579c2cb3b32f9a661e26","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:02.620Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-6bc98aeb1521418d334552a3: done: 0.044779 sec","job_status":"done","queue_duration_s":2.347949,"scheduling_latency_s":2.347997,"redis_calls":12,"redis_duration_s":0.0028669999999999998,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000437,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001267,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001163,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00084061,"exclusive_lock_hold_duration_s":0.004310324,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.044779,"completed_at":"2026-11-12T09:22:05.013Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002436,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 169.982063] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db3d4d8b48fab3b7beb40034","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db3d4d8b48fab3b7beb40034: start","job_status":"start","queue_duration_s":1.467129,"scheduling_latency_s":1.467163}1369gitlab # [ 170.008738] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.042Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db3d4d8b48fab3b7beb40034","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db3d4d8b48fab3b7beb40034: done: 0.02682 sec","job_status":"done","queue_duration_s":1.467129,"scheduling_latency_s":1.467163,"redis_calls":16,"redis_duration_s":0.004383,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000667,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001226,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00249,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015749,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000460953,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02682,"completed_at":"2026-11-12T09:22:05.042Z","load_balancing_strategy":"primary","db_duration_s":0.00064,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 170.028097] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.043Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de7230074fe6861a125e326b","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de7230074fe6861a125e326b: start","job_status":"start","queue_duration_s":1.494903,"scheduling_latency_s":1.494917}1371gitlab # [ 170.039609] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de7230074fe6861a125e326b","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de7230074fe6861a125e326b: done: 0.009608 sec","job_status":"done","queue_duration_s":1.494903,"scheduling_latency_s":1.494917,"redis_calls":13,"redis_duration_s":0.001588,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009608,"completed_at":"2026-11-12T09:22:05.053Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 170.100357] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2d5d56545ae2836ffced603","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2d5d56545ae2836ffced603: start","job_status":"start","queue_duration_s":1.505351,"scheduling_latency_s":1.505365}1373gitlab # [ 170.101051] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2d5d56545ae2836ffced603","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2d5d56545ae2836ffced603: done: 0.009191 sec","job_status":"done","queue_duration_s":1.505351,"scheduling_latency_s":1.505365,"redis_calls":13,"redis_duration_s":0.0015429999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000485,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009191,"completed_at":"2026-11-12T09:22:05.063Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 170.149117] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.064Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2c51f78209383aad8184ace","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2c51f78209383aad8184ace: start","job_status":"start","queue_duration_s":1.515369,"scheduling_latency_s":1.515383}1375gitlab # [ 170.149732] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2c51f78209383aad8184ace","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2c51f78209383aad8184ace: done: 0.009152 sec","job_status":"done","queue_duration_s":1.515369,"scheduling_latency_s":1.515383,"redis_calls":13,"redis_duration_s":0.001504,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005848,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009152,"completed_at":"2026-11-12T09:22:05.073Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 170.162711] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9b673569707ff9349ec0ecf","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9b673569707ff9349ec0ecf: start","job_status":"start","queue_duration_s":1.525345,"scheduling_latency_s":1.525358}1377gitlab # [ 170.200935] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d9b673569707ff9349ec0ecf","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d9b673569707ff9349ec0ecf: done: 0.009353 sec","job_status":"done","queue_duration_s":1.525345,"scheduling_latency_s":1.525358,"redis_calls":13,"redis_duration_s":0.0015279999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009353,"completed_at":"2026-11-12T09:22:05.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 170.206938] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e62e92446a603af9f609e8d","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e62e92446a603af9f609e8d: start","job_status":"start","queue_duration_s":1.535524,"scheduling_latency_s":1.535537}1379gitlab # [ 170.218303] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e62e92446a603af9f609e8d","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e62e92446a603af9f609e8d: done: 0.009016 sec","job_status":"done","queue_duration_s":1.535524,"scheduling_latency_s":1.535537,"redis_calls":13,"redis_duration_s":0.001506,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009016,"completed_at":"2026-11-12T09:22:05.093Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 170.259963] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8eeccbe06d615b7e0abefd9","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8eeccbe06d615b7e0abefd9: start","job_status":"start","queue_duration_s":1.54535,"scheduling_latency_s":1.545364}1381gitlab # [ 170.262583] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8eeccbe06d615b7e0abefd9","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8eeccbe06d615b7e0abefd9: done: 0.009057 sec","job_status":"done","queue_duration_s":1.54535,"scheduling_latency_s":1.545364,"redis_calls":13,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000486,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009057,"completed_at":"2026-11-12T09:22:05.103Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 170.322764] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35b677f5664b0dd538538afc","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35b677f5664b0dd538538afc: start","job_status":"start","queue_duration_s":1.555234,"scheduling_latency_s":1.555246}1383gitlab # [ 170.323918] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.113Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35b677f5664b0dd538538afc","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35b677f5664b0dd538538afc: done: 0.009652 sec","job_status":"done","queue_duration_s":1.555234,"scheduling_latency_s":1.555246,"redis_calls":13,"redis_duration_s":0.002013,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000713,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009652,"completed_at":"2026-11-12T09:22:05.113Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 170.370708] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9541af0dd5dadba522b6e794","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9541af0dd5dadba522b6e794: start","job_status":"start","queue_duration_s":1.565919,"scheduling_latency_s":1.565931}1385gitlab # [ 170.371612] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9541af0dd5dadba522b6e794","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9541af0dd5dadba522b6e794: done: 0.009909 sec","job_status":"done","queue_duration_s":1.565919,"scheduling_latency_s":1.565931,"redis_calls":13,"redis_duration_s":0.002225,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001523,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009909,"completed_at":"2026-11-12T09:22:05.124Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 170.411760] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f29aaa13644d91313a2ae9ef","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f29aaa13644d91313a2ae9ef: start","job_status":"start","queue_duration_s":1.576903,"scheduling_latency_s":1.576917}1387gitlab # [ 170.412665] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f29aaa13644d91313a2ae9ef","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f29aaa13644d91313a2ae9ef: done: 0.015171 sec","job_status":"done","queue_duration_s":1.576903,"scheduling_latency_s":1.576917,"redis_calls":13,"redis_duration_s":0.003062,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015171,"completed_at":"2026-11-12T09:22:05.140Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 170.458816] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de82a4fd9cd2e5eba786448f","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de82a4fd9cd2e5eba786448f: start","job_status":"start","queue_duration_s":1.594073,"scheduling_latency_s":1.594108}1389gitlab # [ 170.459684] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de82a4fd9cd2e5eba786448f","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de82a4fd9cd2e5eba786448f: done: 0.01335 sec","job_status":"done","queue_duration_s":1.594073,"scheduling_latency_s":1.594108,"redis_calls":13,"redis_duration_s":0.001717,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01335,"completed_at":"2026-11-12T09:22:05.156Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 170.501099] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d27323f90d8193b73c94b14","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d27323f90d8193b73c94b14: start","job_status":"start","queue_duration_s":1.608233,"scheduling_latency_s":1.608248}1391gitlab # [ 170.501797] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d27323f90d8193b73c94b14","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d27323f90d8193b73c94b14: done: 0.010925 sec","job_status":"done","queue_duration_s":1.608233,"scheduling_latency_s":1.608248,"redis_calls":13,"redis_duration_s":0.001627,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010925,"completed_at":"2026-11-12T09:22:05.167Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392gitlab # [ 170.540954] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f296ba809169acfccec25f3c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f296ba809169acfccec25f3c: start","job_status":"start","queue_duration_s":1.620156,"scheduling_latency_s":1.620175}1393gitlab # [ 170.541813] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.178Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f296ba809169acfccec25f3c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f296ba809169acfccec25f3c: done: 0.009732 sec","job_status":"done","queue_duration_s":1.620156,"scheduling_latency_s":1.620175,"redis_calls":13,"redis_duration_s":0.001626,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009732,"completed_at":"2026-11-12T09:22:05.178Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 170.593196] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.179Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38956365e019a6d900d14477","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38956365e019a6d900d14477: start","job_status":"start","queue_duration_s":1.630716,"scheduling_latency_s":1.630729}1395gitlab # [ 170.593909] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38956365e019a6d900d14477","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38956365e019a6d900d14477: done: 0.00947 sec","job_status":"done","queue_duration_s":1.630716,"scheduling_latency_s":1.630729,"redis_calls":13,"redis_duration_s":0.0015660000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001039,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00947,"completed_at":"2026-11-12T09:22:05.188Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 170.650582] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"899226add336e7154c85531c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-899226add336e7154c85531c: start","job_status":"start","queue_duration_s":1.641075,"scheduling_latency_s":1.641088}1397gitlab # [ 170.651592] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"899226add336e7154c85531c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-899226add336e7154c85531c: done: 0.009225 sec","job_status":"done","queue_duration_s":1.641075,"scheduling_latency_s":1.641088,"redis_calls":13,"redis_duration_s":0.0015639999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009225,"completed_at":"2026-11-12T09:22:05.198Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 170.666092] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6025f6a1ed792e95f42f6e16","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6025f6a1ed792e95f42f6e16: start","job_status":"start","queue_duration_s":1.651164,"scheduling_latency_s":1.651177}1399gitlab # [ 170.677225] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6025f6a1ed792e95f42f6e16","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6025f6a1ed792e95f42f6e16: done: 0.009392 sec","job_status":"done","queue_duration_s":1.651164,"scheduling_latency_s":1.651177,"redis_calls":13,"redis_duration_s":0.001573,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009392,"completed_at":"2026-11-12T09:22:05.209Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 170.730771] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84d658a5d1cf0fe799c06cf0","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84d658a5d1cf0fe799c06cf0: start","job_status":"start","queue_duration_s":1.661393,"scheduling_latency_s":1.661406}1401gitlab # [ 170.731771] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84d658a5d1cf0fe799c06cf0","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84d658a5d1cf0fe799c06cf0: done: 0.009095 sec","job_status":"done","queue_duration_s":1.661393,"scheduling_latency_s":1.661406,"redis_calls":13,"redis_duration_s":0.001508,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009095,"completed_at":"2026-11-12T09:22:05.219Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 170.759762] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74c0b4cbdc682e6046a02d2c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74c0b4cbdc682e6046a02d2c: start","job_status":"start","queue_duration_s":1.671313,"scheduling_latency_s":1.671328}1403gitlab # [ 170.771550] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74c0b4cbdc682e6046a02d2c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74c0b4cbdc682e6046a02d2c: done: 0.009037 sec","job_status":"done","queue_duration_s":1.671313,"scheduling_latency_s":1.671328,"redis_calls":13,"redis_duration_s":0.001508,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005756,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009037,"completed_at":"2026-11-12T09:22:05.229Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 170.807867] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.229Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21c5b6e1619b3af6a29af98b","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21c5b6e1619b3af6a29af98b: start","job_status":"start","queue_duration_s":1.681188,"scheduling_latency_s":1.681201}1405gitlab # [ 170.856137] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21c5b6e1619b3af6a29af98b","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21c5b6e1619b3af6a29af98b: done: 0.009113 sec","job_status":"done","queue_duration_s":1.681188,"scheduling_latency_s":1.681201,"redis_calls":13,"redis_duration_s":0.001508,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009113,"completed_at":"2026-11-12T09:22:05.238Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 170.857207] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.239Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e58e4c5d09c95a2b389de1c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e58e4c5d09c95a2b389de1c: start","job_status":"start","queue_duration_s":1.691163,"scheduling_latency_s":1.691178}1407gitlab # [ 170.866175] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e58e4c5d09c95a2b389de1c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e58e4c5d09c95a2b389de1c: done: 0.009177 sec","job_status":"done","queue_duration_s":1.691163,"scheduling_latency_s":1.691178,"redis_calls":13,"redis_duration_s":0.001556,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009177,"completed_at":"2026-11-12T09:22:05.249Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 170.914965] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a86ac2b6ef9e915a32e916bb","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a86ac2b6ef9e915a32e916bb: start","job_status":"start","queue_duration_s":1.701174,"scheduling_latency_s":1.701187}1409gitlab # [ 170.915665] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.259Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a86ac2b6ef9e915a32e916bb","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a86ac2b6ef9e915a32e916bb: done: 0.009223 sec","job_status":"done","queue_duration_s":1.701174,"scheduling_latency_s":1.701187,"redis_calls":13,"redis_duration_s":0.001526,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009223,"completed_at":"2026-11-12T09:22:05.259Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 170.956536] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.259Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9086cd9e13f59e90b89413d","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9086cd9e13f59e90b89413d: start","job_status":"start","queue_duration_s":1.711253,"scheduling_latency_s":1.711267}1411gitlab # [ 170.958160] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9086cd9e13f59e90b89413d","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9086cd9e13f59e90b89413d: done: 0.00912 sec","job_status":"done","queue_duration_s":1.711253,"scheduling_latency_s":1.711267,"redis_calls":13,"redis_duration_s":0.001511,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000485,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00912,"completed_at":"2026-11-12T09:22:05.269Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 171.005555] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be241e983476413e9613fd62","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be241e983476413e9613fd62: start","job_status":"start","queue_duration_s":1.721203,"scheduling_latency_s":1.721216}1413gitlab # [ 171.006176] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.279Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be241e983476413e9613fd62","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be241e983476413e9613fd62: done: 0.009111 sec","job_status":"done","queue_duration_s":1.721203,"scheduling_latency_s":1.721216,"redis_calls":13,"redis_duration_s":0.001505,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009111,"completed_at":"2026-11-12T09:22:05.278Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414gitlab # [ 171.039949] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.279Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c0011fc6e362769c1e1ca73","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c0011fc6e362769c1e1ca73: start","job_status":"start","queue_duration_s":1.731276,"scheduling_latency_s":1.731289}1415gitlab # [ 171.082045] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c0011fc6e362769c1e1ca73","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c0011fc6e362769c1e1ca73: done: 0.009486 sec","job_status":"done","queue_duration_s":1.731276,"scheduling_latency_s":1.731289,"redis_calls":13,"redis_duration_s":0.00152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009486,"completed_at":"2026-11-12T09:22:05.289Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 171.122072] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c3731dd56490142d4ab6dc8","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c3731dd56490142d4ab6dc8: start","job_status":"start","queue_duration_s":1.741574,"scheduling_latency_s":1.741608}1417gitlab # [ 171.122836] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c3731dd56490142d4ab6dc8","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c3731dd56490142d4ab6dc8: done: 0.013993 sec","job_status":"done","queue_duration_s":1.741574,"scheduling_latency_s":1.741608,"redis_calls":13,"redis_duration_s":0.005987,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004832,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001155,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013993,"completed_at":"2026-11-12T09:22:05.304Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418gitlab # [ 171.130534] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87325bdd6f37b3106a9f051a","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87325bdd6f37b3106a9f051a: start","job_status":"start","queue_duration_s":1.759572,"scheduling_latency_s":1.759609}1419gitlab # [ 171.175915] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87325bdd6f37b3106a9f051a","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87325bdd6f37b3106a9f051a: done: 0.014904 sec","job_status":"done","queue_duration_s":1.759572,"scheduling_latency_s":1.759609,"redis_calls":13,"redis_duration_s":0.006624,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006424,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014904,"completed_at":"2026-11-12T09:22:05.323Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420gitlab # [ 171.176873] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84301aeebfcf162a2480efcf","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84301aeebfcf162a2480efcf: start","job_status":"start","queue_duration_s":1.778166,"scheduling_latency_s":1.778184}1421gitlab # [ 171.215113] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84301aeebfcf162a2480efcf","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84301aeebfcf162a2480efcf: done: 0.015895 sec","job_status":"done","queue_duration_s":1.778166,"scheduling_latency_s":1.778184,"redis_calls":13,"redis_duration_s":0.0042309999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003645,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015895,"completed_at":"2026-11-12T09:22:05.342Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 171.221165] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.343Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fa74690fdb8e1d03b1d758f","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fa74690fdb8e1d03b1d758f: start","job_status":"start","queue_duration_s":1.79527,"scheduling_latency_s":1.795295}1423gitlab # [ 171.259303] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fa74690fdb8e1d03b1d758f","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fa74690fdb8e1d03b1d758f: done: 0.010471 sec","job_status":"done","queue_duration_s":1.79527,"scheduling_latency_s":1.795295,"redis_calls":13,"redis_duration_s":0.0015569999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010471,"completed_at":"2026-11-12T09:22:05.354Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 171.265943] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65da4815716ee11b45ec76e8","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65da4815716ee11b45ec76e8: start","job_status":"start","queue_duration_s":1.806511,"scheduling_latency_s":1.806524}1425gitlab # [ 171.277540] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65da4815716ee11b45ec76e8","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65da4815716ee11b45ec76e8: done: 0.011141 sec","job_status":"done","queue_duration_s":1.806511,"scheduling_latency_s":1.806524,"redis_calls":13,"redis_duration_s":0.001597,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011141,"completed_at":"2026-11-12T09:22:05.366Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426gitlab # [ 171.330498] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fef29603798a5ddcbe0dd570","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fef29603798a5ddcbe0dd570: start","job_status":"start","queue_duration_s":1.818699,"scheduling_latency_s":1.818713}1427gitlab # [ 171.331208] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fef29603798a5ddcbe0dd570","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fef29603798a5ddcbe0dd570: done: 0.009597 sec","job_status":"done","queue_duration_s":1.818699,"scheduling_latency_s":1.818713,"redis_calls":13,"redis_duration_s":0.001644,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001146,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009597,"completed_at":"2026-11-12T09:22:05.376Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428gitlab # [ 171.358554] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e9b0784a96ba0cc77d98c26","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e9b0784a96ba0cc77d98c26: start","job_status":"start","queue_duration_s":1.829105,"scheduling_latency_s":1.829119}1429gitlab # [ 171.369713] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e9b0784a96ba0cc77d98c26","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e9b0784a96ba0cc77d98c26: done: 0.009621 sec","job_status":"done","queue_duration_s":1.829105,"scheduling_latency_s":1.829119,"redis_calls":13,"redis_duration_s":0.001732,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000658,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009621,"completed_at":"2026-11-12T09:22:05.387Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430gitlab # [ 171.422118] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcc8ca288acf04b9cd1f8974","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcc8ca288acf04b9cd1f8974: start","job_status":"start","queue_duration_s":1.839548,"scheduling_latency_s":1.839562}1431gitlab # [ 171.422874] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcc8ca288acf04b9cd1f8974","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcc8ca288acf04b9cd1f8974: done: 0.009268 sec","job_status":"done","queue_duration_s":1.839548,"scheduling_latency_s":1.839562,"redis_calls":13,"redis_duration_s":0.00148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009268,"completed_at":"2026-11-12T09:22:05.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 171.465627] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"728eb1f45489ed673e94b7fe","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-728eb1f45489ed673e94b7fe: start","job_status":"start","queue_duration_s":1.849658,"scheduling_latency_s":1.849673}1433gitlab # [ 171.466245] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"728eb1f45489ed673e94b7fe","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-728eb1f45489ed673e94b7fe: done: 0.009348 sec","job_status":"done","queue_duration_s":1.849658,"scheduling_latency_s":1.849673,"redis_calls":13,"redis_duration_s":0.001566,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009348,"completed_at":"2026-11-12T09:22:05.407Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434gitlab # [ 171.507798] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c73d505734d0c431b4f3b92","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c73d505734d0c431b4f3b92: start","job_status":"start","queue_duration_s":1.85995,"scheduling_latency_s":1.859964}1435gitlab # [ 171.508704] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c73d505734d0c431b4f3b92","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c73d505734d0c431b4f3b92: done: 0.009286 sec","job_status":"done","queue_duration_s":1.85995,"scheduling_latency_s":1.859964,"redis_calls":13,"redis_duration_s":0.0015279999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000486,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009286,"completed_at":"2026-11-12T09:22:05.417Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436gitlab # [ 171.541634] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b584a8ae046c9b5e830c6d5","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b584a8ae046c9b5e830c6d5: start","job_status":"start","queue_duration_s":1.870237,"scheduling_latency_s":1.870252}1437gitlab # [ 171.583735] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b584a8ae046c9b5e830c6d5","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b584a8ae046c9b5e830c6d5: done: 0.009523 sec","job_status":"done","queue_duration_s":1.870237,"scheduling_latency_s":1.870252,"redis_calls":13,"redis_duration_s":0.0014910000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009523,"completed_at":"2026-11-12T09:22:05.428Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1438gitlab # [ 171.587945] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf9b3cab2d7dff2a0e3784df","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf9b3cab2d7dff2a0e3784df: start","job_status":"start","queue_duration_s":1.880721,"scheduling_latency_s":1.880735}1439gitlab # [ 171.632621] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.481Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf9b3cab2d7dff2a0e3784df","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf9b3cab2d7dff2a0e3784df: done: 0.05188 sec","job_status":"done","queue_duration_s":1.880721,"scheduling_latency_s":1.880735,"redis_calls":13,"redis_duration_s":0.001595,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.05188,"completed_at":"2026-11-12T09:22:05.481Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440gitlab # [ 171.633631] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.482Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1314767c6b3152c1d9fc5c0c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1314767c6b3152c1d9fc5c0c: start","job_status":"start","queue_duration_s":1.93384,"scheduling_latency_s":1.933859}1441gitlab # [ 171.673151] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1314767c6b3152c1d9fc5c0c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1314767c6b3152c1d9fc5c0c: done: 0.014523 sec","job_status":"done","queue_duration_s":1.93384,"scheduling_latency_s":1.933859,"redis_calls":13,"redis_duration_s":0.002046,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001346,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014523,"completed_at":"2026-11-12T09:22:05.496Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442gitlab # [ 171.712286] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.498Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"916279de947ca30ed52f9bbc","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-916279de947ca30ed52f9bbc: start","job_status":"start","queue_duration_s":1.949336,"scheduling_latency_s":1.949352}1443gitlab # [ 171.713027] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"916279de947ca30ed52f9bbc","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-916279de947ca30ed52f9bbc: done: 0.009291 sec","job_status":"done","queue_duration_s":1.949336,"scheduling_latency_s":1.949352,"redis_calls":13,"redis_duration_s":0.0015429999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009291,"completed_at":"2026-11-12T09:22:05.507Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444gitlab # [ 171.725067] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6baca73c09838bee48909669","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6baca73c09838bee48909669: start","job_status":"start","queue_duration_s":1.959951,"scheduling_latency_s":1.959964}1445gitlab # [ 171.736575] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.517Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6baca73c09838bee48909669","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6baca73c09838bee48909669: done: 0.009284 sec","job_status":"done","queue_duration_s":1.959951,"scheduling_latency_s":1.959964,"redis_calls":13,"redis_duration_s":0.001466,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009284,"completed_at":"2026-11-12T09:22:05.517Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 171.803321] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61b81485e4847819e4adc94c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61b81485e4847819e4adc94c: start","job_status":"start","queue_duration_s":1.970039,"scheduling_latency_s":1.970051}1447gitlab # [ 171.804035] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61b81485e4847819e4adc94c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61b81485e4847819e4adc94c: done: 0.011135 sec","job_status":"done","queue_duration_s":1.970039,"scheduling_latency_s":1.970051,"redis_calls":13,"redis_duration_s":0.001478,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000472,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011135,"completed_at":"2026-11-12T09:22:05.529Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 171.852805] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.530Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51310b392eab38219420390f","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51310b392eab38219420390f: start","job_status":"start","queue_duration_s":1.981986,"scheduling_latency_s":1.981999}1449gitlab # [ 171.853717] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51310b392eab38219420390f","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51310b392eab38219420390f: done: 0.00957 sec","job_status":"done","queue_duration_s":1.981986,"scheduling_latency_s":1.981999,"redis_calls":13,"redis_duration_s":0.001601,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00957,"completed_at":"2026-11-12T09:22:05.540Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450gitlab # [ 171.862531] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d77b94d4a74f147b93dc99c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d77b94d4a74f147b93dc99c: start","job_status":"start","queue_duration_s":1.992445,"scheduling_latency_s":1.992458}1451gitlab # [ 171.906239] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d77b94d4a74f147b93dc99c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d77b94d4a74f147b93dc99c: done: 0.00994 sec","job_status":"done","queue_duration_s":1.992445,"scheduling_latency_s":1.992458,"redis_calls":13,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000481,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00994,"completed_at":"2026-11-12T09:22:05.551Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452gitlab # [ 171.907767] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"037b3860c45a27326dfafd81","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-037b3860c45a27326dfafd81: start","job_status":"start","queue_duration_s":2.003215,"scheduling_latency_s":2.003228}1453gitlab # [ 171.951391] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.561Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"037b3860c45a27326dfafd81","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-037b3860c45a27326dfafd81: done: 0.00932 sec","job_status":"done","queue_duration_s":2.003215,"scheduling_latency_s":2.003228,"redis_calls":13,"redis_duration_s":0.001478,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00932,"completed_at":"2026-11-12T09:22:05.561Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454gitlab # [ 171.953195] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb06e32b5ccac5cba3085c82","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb06e32b5ccac5cba3085c82: start","job_status":"start","queue_duration_s":2.013356,"scheduling_latency_s":2.01337}1455gitlab # [ 171.964592] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.571Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb06e32b5ccac5cba3085c82","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb06e32b5ccac5cba3085c82: done: 0.009384 sec","job_status":"done","queue_duration_s":2.013356,"scheduling_latency_s":2.01337,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006359,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009384,"completed_at":"2026-11-12T09:22:05.571Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456gitlab # [ 172.018593] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e25d59cf36d591fa6430b8b2","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e25d59cf36d591fa6430b8b2: start","job_status":"start","queue_duration_s":2.023529,"scheduling_latency_s":2.023543}1457gitlab # [ 172.019463] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e25d59cf36d591fa6430b8b2","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e25d59cf36d591fa6430b8b2: done: 0.009665 sec","job_status":"done","queue_duration_s":2.023529,"scheduling_latency_s":2.023543,"redis_calls":13,"redis_duration_s":0.0017009999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009665,"completed_at":"2026-11-12T09:22:05.581Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 172.067634] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ed606637a19515868f0f573","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ed606637a19515868f0f573: start","job_status":"start","queue_duration_s":2.034604,"scheduling_latency_s":2.034618}1459gitlab # [ 172.068541] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1ed606637a19515868f0f573","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1ed606637a19515868f0f573: done: 0.009776 sec","job_status":"done","queue_duration_s":2.034604,"scheduling_latency_s":2.034618,"redis_calls":13,"redis_duration_s":0.0017430000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009776,"completed_at":"2026-11-12T09:22:05.593Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460gitlab # [ 172.108979] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0b8efe45149fd2a1ef8e168","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0b8efe45149fd2a1ef8e168: start","job_status":"start","queue_duration_s":2.045196,"scheduling_latency_s":2.045209}1461gitlab # [ 172.109248] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.603Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0b8efe45149fd2a1ef8e168","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0b8efe45149fd2a1ef8e168: done: 0.00942 sec","job_status":"done","queue_duration_s":2.045196,"scheduling_latency_s":2.045209,"redis_calls":13,"redis_duration_s":0.0014910000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00942,"completed_at":"2026-11-12T09:22:05.603Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462gitlab # [ 172.156338] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db37dfae759316d2f840441a","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db37dfae759316d2f840441a: start","job_status":"start","queue_duration_s":2.055426,"scheduling_latency_s":2.055438}1463gitlab # [ 172.157026] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db37dfae759316d2f840441a","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db37dfae759316d2f840441a: done: 0.009485 sec","job_status":"done","queue_duration_s":2.055426,"scheduling_latency_s":2.055438,"redis_calls":13,"redis_duration_s":0.001531,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009485,"completed_at":"2026-11-12T09:22:05.613Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464gitlab # [ 172.181546] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.614Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b9da69091fc0e7a5a5d130b","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b9da69091fc0e7a5a5d130b: start","job_status":"start","queue_duration_s":2.065764,"scheduling_latency_s":2.065777}1465gitlab # [ 172.224411] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.623Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b9da69091fc0e7a5a5d130b","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b9da69091fc0e7a5a5d130b: done: 0.009541 sec","job_status":"done","queue_duration_s":2.065764,"scheduling_latency_s":2.065777,"redis_calls":13,"redis_duration_s":0.001578,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009541,"completed_at":"2026-11-12T09:22:05.623Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1466gitlab # [ 172.227996] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f544a1c7d7e64594d0cc237c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f544a1c7d7e64594d0cc237c: start","job_status":"start","queue_duration_s":2.076652,"scheduling_latency_s":2.076665}1467gitlab # [ 172.276159] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f544a1c7d7e64594d0cc237c","created_at":"2026-11-12T09:22:03.438Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-e7d134f0a50a46df","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"505b9d40e7f2ccb91e007bdc526d92eb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:03.548Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f544a1c7d7e64594d0cc237c: done: 0.009531 sec","job_status":"done","queue_duration_s":2.076652,"scheduling_latency_s":2.076665,"redis_calls":13,"redis_duration_s":0.001526,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009531,"completed_at":"2026-11-12T09:22:05.634Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468gitlab # [ 172.277064] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"3c502569e061c29e0e5d811a","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-3c502569e061c29e0e5d811a: start","job_status":"start","queue_duration_s":0.737508,"scheduling_latency_s":0.737523}1469gitlab # [ 172.315243] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"3c502569e061c29e0e5d811a","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-3c502569e061c29e0e5d811a: done: 0.072722 sec","job_status":"done","queue_duration_s":0.737508,"scheduling_latency_s":0.737523,"redis_calls":16,"redis_duration_s":0.011725,"redis_read_bytes":14,"redis_write_bytes":1526,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000308,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003431,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":423,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007986,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":802,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017332,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000982527,"duration_s":0.072722,"completed_at":"2026-11-12T09:22:05.708Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.040635,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1470gitlab # [ 172.321793] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"e416cfdfcf0b894159f48afa","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-e416cfdfcf0b894159f48afa: start","job_status":"start","queue_duration_s":0.813358,"scheduling_latency_s":0.813372}1471gitlab # [ 172.333830] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"e416cfdfcf0b894159f48afa","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-e416cfdfcf0b894159f48afa: done: 0.014269 sec","job_status":"done","queue_duration_s":0.813358,"scheduling_latency_s":0.813372,"redis_calls":13,"redis_duration_s":0.0032129999999999997,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001916,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001297,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014269,"completed_at":"2026-11-12T09:22:05.725Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000656,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472gitlab # [ 172.391155] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8b9c9999880578a713e23c0b","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8b9c9999880578a713e23c0b: start","job_status":"start","queue_duration_s":0.828481,"scheduling_latency_s":0.828495}1473gitlab # [ 172.391854] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8b9c9999880578a713e23c0b","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8b9c9999880578a713e23c0b: done: 0.011355 sec","job_status":"done","queue_duration_s":0.828481,"scheduling_latency_s":0.828495,"redis_calls":13,"redis_duration_s":0.001614,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007248,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011355,"completed_at":"2026-11-12T09:22:05.738Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00112,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1474gitlab # [ 172.432616] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"4f9311661c6bf581ab5e2c4e","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-4f9311661c6bf581ab5e2c4e: start","job_status":"start","queue_duration_s":0.840666,"scheduling_latency_s":0.840681}1475gitlab # [ 172.432783] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"4f9311661c6bf581ab5e2c4e","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-4f9311661c6bf581ab5e2c4e: done: 0.010618 sec","job_status":"done","queue_duration_s":0.840666,"scheduling_latency_s":0.840681,"redis_calls":13,"redis_duration_s":0.001509,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000483,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010618,"completed_at":"2026-11-12T09:22:05.749Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00052,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476gitlab # [ 172.477685] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"3e8b1505c7da258e4632cf04","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-3e8b1505c7da258e4632cf04: start","job_status":"start","queue_duration_s":0.852141,"scheduling_latency_s":0.852154}1477gitlab # [ 172.478660] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"3e8b1505c7da258e4632cf04","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-3e8b1505c7da258e4632cf04: done: 0.010294 sec","job_status":"done","queue_duration_s":0.852141,"scheduling_latency_s":0.852154,"redis_calls":13,"redis_duration_s":0.001545,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010294,"completed_at":"2026-11-12T09:22:05.760Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000485,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478gitlab # [ 172.509795] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.761Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"2afd1d27723ab49ad50cb892","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-2afd1d27723ab49ad50cb892: start","job_status":"start","queue_duration_s":0.863288,"scheduling_latency_s":0.863301}1479gitlab # [ 172.552781] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.781Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"2afd1d27723ab49ad50cb892","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-2afd1d27723ab49ad50cb892: done: 0.020487 sec","job_status":"done","queue_duration_s":0.863288,"scheduling_latency_s":0.863301,"redis_calls":13,"redis_duration_s":0.010464999999999999,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002152,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008313,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020487,"completed_at":"2026-11-12T09:22:05.781Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002011,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1480gitlab # [ 172.556822] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"6dd80658a3b65c379470e2c8","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-6dd80658a3b65c379470e2c8: start","job_status":"start","queue_duration_s":0.886433,"scheduling_latency_s":0.886448}1481gitlab # [ 172.569049] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"6dd80658a3b65c379470e2c8","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-6dd80658a3b65c379470e2c8: done: 0.011376 sec","job_status":"done","queue_duration_s":0.886433,"scheduling_latency_s":0.886448,"redis_calls":13,"redis_duration_s":0.001632,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001135,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011376,"completed_at":"2026-11-12T09:22:05.795Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000426,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482gitlab # [ 172.624769] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"6eb09d4ba0cbdca785f82aa3","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-6eb09d4ba0cbdca785f82aa3: start","job_status":"start","queue_duration_s":0.898928,"scheduling_latency_s":0.898942}1483gitlab # [ 172.625663] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"6eb09d4ba0cbdca785f82aa3","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-6eb09d4ba0cbdca785f82aa3: done: 0.010431 sec","job_status":"done","queue_duration_s":0.898928,"scheduling_latency_s":0.898942,"redis_calls":13,"redis_duration_s":0.0015760000000000001,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010431,"completed_at":"2026-11-12T09:22:05.807Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000642,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484gitlab # [ 172.686177] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8f19150745828b7eedaecd70","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8f19150745828b7eedaecd70: start","job_status":"start","queue_duration_s":0.910821,"scheduling_latency_s":0.910836}1485gitlab # [ 172.686871] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8f19150745828b7eedaecd70","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8f19150745828b7eedaecd70: done: 0.010251 sec","job_status":"done","queue_duration_s":0.910821,"scheduling_latency_s":0.910836,"redis_calls":13,"redis_duration_s":0.001572,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001082,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010251,"completed_at":"2026-11-12T09:22:05.819Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1486gitlab # [ 172.697715] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"1f7a94947d1a3f344e829e53","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-1f7a94947d1a3f344e829e53: start","job_status":"start","queue_duration_s":0.921884,"scheduling_latency_s":0.921898}1487gitlab # [ 172.710095] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:05.830Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"1f7a94947d1a3f344e829e53","created_at":"2026-11-12T09:22:04.871Z","trace_propagation_headers":{"sentry-trace":"c27e66c7c190460696a841c01eba5133-67472563d29f464b","baggage":"sentry-trace_id=c27e66c7c190460696a841c01eba5133,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"c5f2f0abbb8dd0a51394253d5cf5236a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:04.898Z","job_size_bytes":2,"pid":1640,"sidekiq_tid":"maf0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-1f7a94947d1a3f344e829e53: done: 0.010012 sec","job_status":"done","queue_duration_s":0.921884,"scheduling_latency_s":0.921898,"redis_calls":13,"redis_duration_s":0.001521,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000483,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010012,"completed_at":"2026-11-12T09:22:05.830Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000366,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488gitlab # [ 172.930812] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2501624034 wd_nsec: 25016239221489gitlab # [ 185.254963] postgres[2438]: [2438] ERROR: relation "table_without_models" does not exist1490gitlab # [ 185.256248] postgres[2438]: [2438] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1491gitlab # [ 188.108879] runuser[2227]: pam_unix(runuser:session): session closed for user gitlab1492gitlab # [ 188.112420] setup-gitlab-start[2202]: + systemctl restart gitlab-sidekiq.service1493gitlab # [ 188.134416] systemd[1]: Stopping gitlab-sidekiq.service...1494gitlab # [ 188.136969] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:23.168Z","message":"Shutting down"}1495gitlab # [ 188.140219] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:23.168Z","message":"Scheduler exiting..."}1496gitlab # [ 188.143531] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:23.170Z","message":"Terminating quiet threads for default capsule"}1497gitlab # [ 188.147376] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:23.170Z","message":"Scheduler exiting..."}1498gitlab # [ 188.638266] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:23.671Z","message":"Pausing to allow jobs to finish..."}1499gitlab # [ 191.142602] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:26.175Z","message":"Bye!"}1500gitlab # [ 191.168204] sidekiq-cluster[1640]: {"severity":"INFO","time":"2026-11-12T09:22:26.201Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1640,"worker_id":"sidekiq_0","memwd_rss_bytes":941072384,"retry":0}1501gitlab # [ 191.700546] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1502gitlab # [ 191.703081] systemd[1]: Stopped gitlab-sidekiq.service.1503gitlab # [ 191.704997] systemd[1]: gitlab-sidekiq.service: Consumed 1min 610ms CPU time over 1min 55.288s wall clock time, 957.1M memory peak, 4.4M read from disk.1504gitlab # [ 191.709753] systemd[1]: Started gitlab-sidekiq.service.1505gitlab # [ 191.712297] setup-gitlab-start[2202]: + mkdir -p /var/lib/secrets1506gitlab # [ 191.714388] setup-gitlab-start[2202]: + echo nixbot-test-token-of-20-chars1507gitlab # [ 191.716667] setup-gitlab-start[2202]: + chmod 644 /var/lib/secrets/gitlab-token1508gitlab # [ 191.719158] setup-gitlab-start[2202]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1509gitlab # [ 191.977300] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:27 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 2311510gitlab # [ 192.535549] gitaly[899]: time="2026-11-12T09:22:27.568Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-12T09:22:37.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.567" pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1511gitlab # [ 192.559182] gitaly[899]: time="2026-11-12T09:22:27.591Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-12T09:22:37.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.567" grpc.time_ms=1.722 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1512gitlab # [ 192.684707] gitaly[899]: time="2026-11-12T09:22:27.718Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-12T09:22:37.157" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.718" pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1513gitlab # [ 192.707709] gitaly[899]: time="2026-11-12T09:22:27.719Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-12T09:22:37.157" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.718" grpc.time_ms=0.38 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1514gitlab # [ 192.734290] gitaly[899]: time="2026-11-12T09:22:27.768Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK diskcache=ced89b60-cc95-4482-8dc7-3afcb7e98bfe grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-12T09:22:37.163" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.724" pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1515gitlab # [ 192.752671] gitaly[899]: time="2026-11-12T09:22:27.768Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1579 command.oublock=32 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-12T09:22:37.163" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.724" grpc.time_ms=43.63 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1516gitlab # [ 192.766433] gitaly[899]: time="2026-11-12T09:22:27.778Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-12T09:22:37.216" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.777" pid=899 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1517gitlab # [ 192.775996] gitaly[899]: time="2026-11-12T09:22:27.778Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-12T09:22:37.216" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.777" grpc.time_ms=1.459 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1518gitlab # [ 192.785813] gitaly[899]: time="2026-11-12T09:22:27.787Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-12T09:22:37.221" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.781" grpc.time_ms=5.995 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1519gitlab # [ 192.816784] gitaly[899]: time="2026-11-12T09:22:27.850Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-12T09:22:37.290" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.850" pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1520gitlab # [ 192.817667] gitaly[899]: time="2026-11-12T09:22:27.851Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-12T09:22:37.290" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.850" grpc.time_ms=0.226 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1521gitlab # [ 192.874275] gitaly[899]: time="2026-11-12T09:22:27.908Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK diskcache=8c51ac00-8a0d-468b-ba16-8139bae51f25 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-12T09:22:37.001" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.872" pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1522gitlab # [ 192.875357] gitaly[899]: time="2026-11-12T09:22:27.908Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1580 command.oublock=32 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-12T09:22:37.001" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.872" grpc.time_ms=36.442 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1523gitlab # [ 192.883523] gitaly[899]: time="2026-11-12T09:22:27.917Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-12T09:22:37.046" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.917" pid=899 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1524gitlab # [ 192.884305] gitaly[899]: time="2026-11-12T09:22:27.917Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-12T09:22:37.046" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.917" grpc.time_ms=0.619 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1525gitlab # [ 192.893146] gitaly[899]: time="2026-11-12T09:22:27.926Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-12T09:22:37.050" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.921" grpc.time_ms=5.555 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1526gitlab # [ 192.915284] gitaly[899]: time="2026-11-12T09:22:27.949Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-12T09:22:37.078" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.949" grpc.time_ms=0.229 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1527gitlab # [ 192.920326] gitaly[899]: time="2026-11-12T09:22:27.954Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-12T09:22:37.083" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:27.954" grpc.time_ms=0.312 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1528gitlab # [ 193.438535] gitaly[899]: time="2026-11-12T09:22:28.472Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1K2D9EFHSCYHGSM3HSWK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-12T09:22:38.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:28.461" grpc.time_ms=11.229 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1529gitlab # [ 193.687466] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:28 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 16981530gitlab # [ 193.694171] setup-gitlab-start[2493]: {"id":1,"message":null,"error":null}1531gitlab # [ 194.847712] sidekiq-cluster[2477]: {"severity":"INFO","time":"2026-11-12T09:22:29.863Z","message":"Starting cluster with 1 processes"}1532gitlab # [ 199.322961] gitaly[899]: time="2026-11-12T09:22:34.356Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1SMWM91AXSWRETGAWPQZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-12T09:22:44.005" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:34.345" grpc.time_ms=10.382 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1533gitlab # [ 199.850806] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:34 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 11261534gitlab # [ 199.855608] setup-gitlab-start[2202]: + mkdir -p /tmp/test-flake1535gitlab # [ 199.858311] setup-gitlab-start[2202]: + cd /tmp/test-flake1536gitlab # [ 199.859412] setup-gitlab-start[2202]: + git init -b master1537gitlab # [ 199.886396] setup-gitlab-start[2574]: Initialized empty Git repository in /tmp/test-flake/.git/1538gitlab # [ 199.887650] setup-gitlab-start[2202]: + git config user.name test1539gitlab # [ 199.891718] setup-gitlab-start[2202]: + git config user.email test@example.com1540gitlab # [ 199.896455] setup-gitlab-start[2202]: + cat1541gitlab # [ 199.899128] setup-gitlab-start[2202]: + git add flake.nix1542gitlab # [ 199.902503] setup-gitlab-start[2202]: + git commit -m 'initial commit'1543gitlab # [ 199.928815] setup-gitlab-start[2579]: [master (root-commit) 85399d3] initial commit1544gitlab # [ 199.929878] setup-gitlab-start[2579]: 1 file changed, 10 insertions(+)1545gitlab # [ 199.931079] setup-gitlab-start[2579]: create mode 100644 flake.nix1546gitlab # [ 199.932200] setup-gitlab-start[2202]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1547gitlab # [ 199.938220] setup-gitlab-start[2202]: + git push -u origin master1548gitlab # [ 199.995400] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:35 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 321549gitlab # [ 200.091660] gitaly[899]: time="2026-11-12T09:22:35.125Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M77H1TWQ0S5M1MQYA1XB5VT6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:35.115" grpc.time_ms=10.172 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1550gitlab # [ 200.104106] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:35 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 961551gitlab # [ 200.366752] gitaly[899]: time="2026-11-12T09:22:35.399Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-12T09:22:45.047" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:35.388" grpc.time_ms=11.271 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1552gitlab # [ 200.429301] gitaly[899]: time="2026-11-12T09:22:35.463Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-12T09:23:05.053" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-12T09:22:35.453" grpc.time_ms=9.443 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1553gitlab # [ 200.483512] gitaly[899]: time="2026-11-12T09:22:35.517Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC duration_ms=221ns method=POST pid=899 status=200 url="http://unix/api/v4/internal/allowed"1554gitlab # [ 200.525560] gitaly[899]: time="2026-11-12T09:22:35.559Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC duration_ms=35ns method=POST pid=899 status=200 url="http://unix/api/v4/internal/pre_receive"1555gitlab # [ 200.536653] gitaly[899]: time="2026-11-12T09:22:35.560Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:22:35.294" grpc.time_ms=265.073 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1556gitlab # [ 200.575471] gitaly[899]: time="2026-11-12T09:22:35.609Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:22:35.607" grpc.time_ms=1.341 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1557gitlab # [ 200.627245] gitaly[899]: time="2026-11-12T09:22:35.661Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:22:35.660" grpc.time_ms=0.445 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1558gitlab # [ 200.667383] gitaly[899]: time="2026-11-12T09:22:35.701Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:22:35.700" grpc.time_ms=0.508 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1559gitlab # [ 200.798272] gitaly[899]: time="2026-11-12T09:22:35.832Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-12T09:22:45.009" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:35.820" grpc.time_ms=11.661 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1560gitlab # [ 200.819505] gitaly[899]: time="2026-11-12T09:22:35.853Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1580 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-12T09:22:45.028" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:22:35.838" grpc.time_ms=14.548 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1561gitlab # [ 200.830228] gitaly[899]: time="2026-11-12T09:22:35.863Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC duration_ms=121ns method=POST pid=899 status=200 url="http://unix/api/v4/internal/post_receive"1562gitlab # [ 200.832709] gitaly[899]: time="2026-11-12T09:22:35.864Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:22:35.741" grpc.time_ms=122.444 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1563gitlab # [ 200.876391] gitaly[899]: time="2026-11-12T09:22:35.910Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC diskcache=ce3edbe8-fa8b-4b5c-a71e-35f998452ac4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:35.224" pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1564gitlab # [ 200.879331] gitlab-workhorse[825]: time="2026-11-12T09:22:35Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521565gitlab # [ 200.880316] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:35 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 7631566gitlab # [ 200.881646] setup-gitlab-start[2583]: To http://gitlab/root/test-flake.git1567gitlab # [ 200.882662] setup-gitlab-start[2583]: * [new branch] master -> master1568gitlab # [ 200.883709] gitaly[899]: time="2026-11-12T09:22:35.910Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=198 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=901120 command.minflt=13553 command.oublock=32 command.real_time_ms=673 command.spawn_token_fork_ms=0 command.system_time_ms=129 command.user_time_ms=69 component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:35.224" grpc.time_ms=685.771 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1569gitlab # [ 200.904531] setup-gitlab-start[2583]: branch 'master' set up to track 'origin/master'.1570gitlab # [ 200.909430] systemd[1]: Finished setup-gitlab.service.1571gitlab # [ 200.917451] systemd[1]: Starting nixbot CI service...1572gitlab: (finished: waiting for unit setup-gitlab.service, in 201.89 seconds)1573gitlab: waiting for unit nixbot.service1574gitlab # [ 202.243220] nixbot[2660]: {"ts": "2026-11-12T09:22:37.276Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1575gitlab # [ 202.277319] nixbot[2660]: {"ts": "2026-11-12T09:22:37.311Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1576gitlab # [ 202.304927] nixbot[2660]: {"ts": "2026-11-12T09:22:37.338Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1577gitlab # [ 202.315592] nixbot[2660]: {"ts": "2026-11-12T09:22:37.344Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1578gitlab # [ 202.321105] nixbot[2660]: {"ts": "2026-11-12T09:22:37.351Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1579gitlab # [ 202.327919] nixbot[2660]: {"ts": "2026-11-12T09:22:37.361Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1580gitlab # [ 202.334385] nixbot[2660]: {"ts": "2026-11-12T09:22:37.368Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1581gitlab # [ 202.337140] nixbot[2660]: {"ts": "2026-11-12T09:22:37.371Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1582gitlab # [ 202.344612] nixbot[2660]: {"ts": "2026-11-12T09:22:37.378Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1583gitlab # [ 202.351627] nixbot[2660]: {"ts": "2026-11-12T09:22:37.385Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1584gitlab # [ 202.358403] nixbot[2660]: {"ts": "2026-11-12T09:22:37.392Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1585gitlab # [ 202.365130] nixbot[2660]: {"ts": "2026-11-12T09:22:37.399Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1586gitlab # [ 202.370288] nixbot[2660]: {"ts": "2026-11-12T09:22:37.402Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1587gitlab # [ 202.375128] nixbot[2660]: {"ts": "2026-11-12T09:22:37.405Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1588gitlab # [ 202.379712] nixbot[2660]: {"ts": "2026-11-12T09:22:37.413Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1589gitlab # [ 202.386910] nixbot[2660]: {"ts": "2026-11-12T09:22:37.420Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1590gitlab # [ 202.393503] nixbot[2660]: {"ts": "2026-11-12T09:22:37.427Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1591gitlab # [ 202.400241] nixbot[2660]: {"ts": "2026-11-12T09:22:37.434Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1592gitlab # [ 202.411283] nixbot[2660]: {"ts": "2026-11-12T09:22:37.445Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1593gitlab # [ 202.414942] nixbot[2660]: {"ts": "2026-11-12T09:22:37.448Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1594gitlab # [ 202.422673] nixbot[2660]: {"ts": "2026-11-12T09:22:37.456Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1595gitlab # [ 202.434538] nixbot[2660]: {"ts": "2026-11-12T09:22:37.468Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1596gitlab # [ 202.438175] nixbot[2660]: {"ts": "2026-11-12T09:22:37.472Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1597gitlab # [ 202.442286] nixbot[2660]: {"ts": "2026-11-12T09:22:37.476Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1598gitlab # [ 202.445947] nixbot[2660]: {"ts": "2026-11-12T09:22:37.479Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1599gitlab # [ 202.450305] nixbot[2660]: {"ts": "2026-11-12T09:22:37.484Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1600gitlab # [ 202.455249] nixbot[2660]: {"ts": "2026-11-12T09:22:37.489Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1601gitlab # [ 202.459756] nixbot[2660]: {"ts": "2026-11-12T09:22:37.493Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1602gitlab # [ 202.464118] nixbot[2660]: {"ts": "2026-11-12T09:22:37.498Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1603gitlab # [ 202.484167] nixbot[2660]: {"ts": "2026-11-12T09:22:37.518Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1604gitlab # [ 202.492103] nixbot[2660]: {"ts": "2026-11-12T09:22:37.525Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1605gitlab # [ 202.499365] nixbot[2660]: {"ts": "2026-11-12T09:22:37.533Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1606gitlab # [ 202.505738] nixbot[2660]: {"ts": "2026-11-12T09:22:37.539Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1607gitlab # [ 202.515334] nixbot[2660]: {"ts": "2026-11-12T09:22:37.549Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1608gitlab # [ 202.519927] nixbot[2660]: {"ts": "2026-11-12T09:22:37.553Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1609gitlab # [ 202.529827] nixbot[2660]: {"ts": "2026-11-12T09:22:37.563Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1610gitlab # [ 202.535371] nixbot[2660]: {"ts": "2026-11-12T09:22:37.569Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1611gitlab # [ 202.541567] nixbot[2660]: {"ts": "2026-11-12T09:22:37.575Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "effect_when_changed"}1612gitlab # [ 202.546724] nixbot[2660]: {"ts": "2026-11-12T09:22:37.580Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 38, "migration": "commit_statuses"}1613gitlab # [ 202.932887] nixbot[2660]: {"ts": "2026-11-12T09:22:37.966Z", "level": "info", "logger": "uvicorn.error", "msg": "Started server process [2660]", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"}1614gitlab # [ 202.934715] nixbot[2660]: {"ts": "2026-11-12T09:22:37.967Z", "level": "info", "logger": "uvicorn.error", "msg": "Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)"}1615gitlab # [ 202.952394] nixbot[2660]: {"ts": "2026-11-12T09:22:37.986Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:43608 - \"GET /health HTTP/1.1\" 200"}1616gitlab # [ 202.960946] systemd[1]: Started nixbot CI service.1617gitlab # [ 202.962545] systemd[1]: Reached target Multi-User System.1618gitlab # [ 202.964554] systemd[1]: Startup finished in 1.242s (kernel) + 3.687s (initrd) + 3min 18.033s (userspace) = 3min 22.964s.1619gitlab # [ 203.205517] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:38 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2531620gitlab # [ 203.238812] nixbot[2660]: {"ts": "2026-11-12T09:22:38.272Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1621gitlab # [ 203.321742] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 661622gitlab # [ 203.330721] nixbot[2660]: {"ts": "2026-11-12T09:22:38.363Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1623gitlab: (finished: waiting for unit nixbot.service, in 2.10 seconds)1624gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1625gitlab # [ 203.411430] nixbot[2660]: {"ts": "2026-11-12T09:22:38.445Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:49824 - \"GET /health HTTP/1.1\" 200"}1626gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1627subtest: project discovered and webhook registered1628gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1629gitlab # [ 203.494197] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 1561630gitlab # [ 203.527549] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 741631gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.11 seconds)1632(finished: subtest: project discovered and webhook registered, in 0.11 seconds)1633subtest: push delivers the webhook1634gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1635gitlab # [ 203.679861] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:38 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1711636gitlab # [ 203.735335] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 381637gitlab # [ 203.785564] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:38 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 441638gitlab # [ 203.878085] gitaly[899]: time="2026-11-12T09:22:38.911Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1YKBWVPB47AAVA561MD4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-11-12T09:23:08.095" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:22:38.896" grpc.time_ms=14.206 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1639gitlab # [ 203.927465] gitaly[899]: time="2026-11-12T09:22:38.961Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1YKBWVPB47AAVA561MD4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:23:08.159" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:22:38.960" grpc.time_ms=0.847 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1640gitlab # [ 203.944861] gitaly[899]: time="2026-11-12T09:22:38.978Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1YKBWVPB47AAVA561MD4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-11-12T09:22:48.096" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:22:38.967" grpc.time_ms=11.564 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1641gitlab # [ 204.004633] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 2111642gitlab # [ 204.090689] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 761643gitlab # [ 204.102039] nixbot[2660]: {"ts": "2026-11-12T09:22:39.136Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1644gitlab # [ 204.254824] gitaly[899]: time="2026-11-12T09:22:39.288Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1YZWCZKRFZKGS7XY4NF6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:39.275" grpc.time_ms=12.899 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1645gitlab # [ 204.284646] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 891646gitlab # [ 204.404738] gitaly[899]: time="2026-11-12T09:22:39.438Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1Z2VXC9FDTJ604X1K0YX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:39.425" pid=899 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1647gitlab # [ 204.405696] gitaly[899]: time="2026-11-12T09:22:39.438Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1Z2VXC9FDTJ604X1K0YX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:39.425" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1648gitlab # [ 204.406726] gitaly[899]: time="2026-11-12T09:22:39.440Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1Z2VXC9FDTJ604X1K0YX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:39.425" grpc.time_ms=13.567 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1649gitlab # [ 204.407832] gitlab-workhorse[825]: time="2026-11-12T09:22:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131650gitlab # [ 204.409377] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1181651gitlab # [ 204.576162] gitaly[899]: time="2026-11-12T09:22:39.610Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1Z6QCJG2S1Y78FFJ58GH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:39.579" pid=899 remote_ip="2001:db8:1::1" request_sha=30ec9c0df0af828c7a6d73b1331961da8463cfa4 response_bytes=228 span.kind=server system=grpc1652gitlab # [ 204.577063] gitaly[899]: time="2026-11-12T09:22:39.610Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1204 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1Z6QCJG2S1Y78FFJ58GH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:39.579" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1653gitlab # [ 204.577721] gitaly[899]: time="2026-11-12T09:22:39.611Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1204 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1Z6QCJG2S1Y78FFJ58GH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:39.579" grpc.time_ms=31.449 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1654gitlab # [ 204.578867] gitlab-workhorse[825]: time="2026-11-12T09:22:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2281655gitlab # [ 204.579964] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 228 "" "git/2.55.0" 1641656gitlab # [ 204.724054] gitaly[899]: time="2026-11-12T09:22:39.758Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1ZEKFGVAC0GGP9F4EYEW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:39.749" grpc.time_ms=8.04 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1657gitlab # [ 204.752080] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 861658gitlab # [ 204.820482] gitaly[899]: time="2026-11-12T09:22:39.854Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1ZHFDZ0WM9N2T735DVT2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:39.841" pid=899 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1659gitlab # [ 204.821733] gitaly[899]: time="2026-11-12T09:22:39.854Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1ZHFDZ0WM9N2T735DVT2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:39.841" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1660gitlab # [ 204.822762] gitlab-workhorse[825]: time="2026-11-12T09:22:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671661gitlab # [ 204.823856] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 631662gitlab # [ 204.824964] gitaly[899]: time="2026-11-12T09:22:39.855Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1ZHFDZ0WM9N2T735DVT2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:39.841" grpc.time_ms=13.118 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1663gitlab # [ 204.981258] gitaly[899]: time="2026-11-12T09:22:40.014Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1ZPH9XPJCFEY35MP231S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:40.003" grpc.time_ms=10.411 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1664gitlab # [ 204.983421] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 611665gitlab # [ 205.106090] gitaly[899]: time="2026-11-12T09:22:40.140Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1ZSFTKYQR6P3C7ES2XZZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:40.119" pid=899 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1666gitlab # [ 205.107421] gitaly[899]: time="2026-11-12T09:22:40.140Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1164 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1ZSFTKYQR6P3C7ES2XZZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:40.119" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1667gitlab # [ 205.108063] gitaly[899]: time="2026-11-12T09:22:40.141Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1164 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1ZSFTKYQR6P3C7ES2XZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:40.119" grpc.time_ms=21.814 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1668gitlab # [ 205.108780] gitlab-workhorse[825]: time="2026-11-12T09:22:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091669gitlab # [ 205.109883] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 941670gitlab # [ 205.266419] gitaly[899]: time="2026-11-12T09:22:40.300Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:23:10.098" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:22:40.299" grpc.time_ms=0.711 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1671gitlab # [ 205.331709] gitaly[899]: time="2026-11-12T09:22:40.365Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-12T09:23:10.153" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:22:40.354" grpc.time_ms=11.493 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1672gitlab # [ 205.393065] gitaly[899]: time="2026-11-12T09:22:40.427Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:22:50.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:22:40.413" grpc.time_ms=12.692 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1673gitlab # [ 205.448070] gitaly[899]: time="2026-11-12T09:22:40.482Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:22:50.060" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:22:40.470" grpc.time_ms=11.243 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1674gitlab # [ 205.514237] gitaly[899]: time="2026-11-12T09:22:40.548Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-12T09:22:50.136" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:40.547" grpc.time_ms=0.405 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1675gitlab # [ 205.555292] gitaly[899]: time="2026-11-12T09:22:40.589Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-12T09:22:50.166" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:22:40.577" grpc.time_ms=11.434 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1676gitlab # [ 205.599797] gitaly[899]: time="2026-11-12T09:22:40.633Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:22:40.633" grpc.time_ms=0.322 pid=899 span.kind=server system=grpc1677gitlab # [ 205.641148] gitaly[899]: time="2026-11-12T09:22:40.673Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:22:40.672" grpc.time_ms=0.342 pid=899 span.kind=server system=grpc1678gitlab # [ 205.659096] gitaly[899]: time="2026-11-12T09:22:40.693Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP diskcache=fa8f552f-402f-44e9-badb-d031aaf7f24a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-12T09:22:50.181" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:40.592" pid=899 ref=refs/keep-around/85399d301bf194486bc26d409ef791d202275c25 remote_ip="2001:db8:1::1" revision=85399d301bf194486bc26d409ef791d202275c25 span.kind=server system=grpc username=root1679gitlab # [ 205.659621] gitaly[899]: time="2026-11-12T09:22:40.693Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=60 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5209 command.oublock=8 command.real_time_ms=91 command.spawn_token_fork_ms=1 command.system_time_ms=39 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-12T09:22:50.181" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:22:40.592" grpc.time_ms=101.28 pid=899 ref=refs/keep-around/85399d301bf194486bc26d409ef791d202275c25 remote_ip="2001:db8:1::1" revision=85399d301bf194486bc26d409ef791d202275c25 span.kind=server system=grpc username=root1680gitlab # [ 205.663382] gitaly[899]: time="2026-11-12T09:22:40.697Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:23:10.496" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:22:40.697" grpc.time_ms=0.327 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1681gitlab # [ 205.925226] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/85399d301bf194486bc26d409ef791d202275c25 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 7131682gitlab # [ 205.946932] nixbot[2660]: {"ts": "2026-11-12T09:22:40.980Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-85399d301bf1-0"}1683gitlab # [ 206.151882] nixbot[2660]: {"ts": "2026-11-12T09:22:41.185Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-85399d301bf1-0", "argv0": "bwrap"}1684gitlab # [ 206.247412] systemd[1]: Started Nix Daemon.1685gitlab # [ 206.324658] nix-daemon[2825]: accepted connection from pid 2820, user nixbot1686gitlab # [ 210.902944] nix-daemon[2825]: accepted connection from pid 2872, user nixbot1687gitlab # [ 210.924079] nix-daemon[2825]: accepted connection from pid 2820, user nixbot1688gitlab # [ 215.242404] nix-daemon[2825]: accepted connection from pid 2820, user nixbot1689gitlab # [ 215.338269] nix-daemon[2825]: accepted connection from pid 2917, user nixbot1690gitlab # [ 215.373759] gitaly[899]: time="2026-11-12T09:22:50.407Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H29VHPB1R29ZGCS3TV1D4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:23:20.005" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:22:50.406" grpc.time_ms=0.736 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1691gitlab # [ 215.478176] gitaly[899]: time="2026-11-12T09:22:50.512Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M77H29VHPB1R29ZGCS3TV1D4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:00.160" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:22:50.501" grpc.time_ms=10.947 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1692gitlab # [ 215.603585] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:50 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/85399d301bf194486bc26d409ef791d202275c25 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2821693gitlab # [ 215.686053] gitaly[899]: time="2026-11-12T09:22:50.719Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H2A4PK75NA23JBRTNDY9S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:23:20.518" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:22:50.718" grpc.time_ms=0.777 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1694gitlab # [ 215.737632] gitaly[899]: time="2026-11-12T09:22:50.771Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M77H2A4PK75NA23JBRTNDY9S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:00.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:22:50.762" grpc.time_ms=8.666 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1695gitlab # [ 215.825903] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:50 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/85399d301bf194486bc26d409ef791d202275c25 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2121696gitlab # [ 220.012801] nix-daemon[2825]: accepted connection from pid 2985, user nixbot1697gitlab # [ 220.092410] gitaly[899]: time="2026-11-12T09:22:55.125Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H2EF6D2QFS3KE0042V6BM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:23:25.024" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:22:55.124" grpc.time_ms=0.696 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1698gitlab # [ 220.150500] gitaly[899]: time="2026-11-12T09:22:55.184Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M77H2EF6D2QFS3KE0042V6BM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:05.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:22:55.173" grpc.time_ms=11.051 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1699gitlab # [ 220.223451] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/85399d301bf194486bc26d409ef791d202275c25 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1771700gitlab # [ 220.465277] nixbot[2660]: {"ts": "2026-11-12T09:22:55.498Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 1, "sha": "85399d301bf194486bc26d409ef791d202275c25", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []}1701gitlab # [ 220.585132] gitaly[899]: time="2026-11-12T09:22:55.619Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H2EXZ95AJM7SYKD9YZTDE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:55.605" grpc.time_ms=13.537 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1702gitlab # [ 220.613376] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:55 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 681703gitlab # [ 220.677465] gitaly[899]: time="2026-11-12T09:22:55.711Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M77H2F14AVD2DMQY0ZDNJCTE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:55.699" pid=899 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1704gitlab # [ 220.700463] gitaly[899]: time="2026-11-12T09:22:55.733Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M77H2F14AVD2DMQY0ZDNJCTE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:55.699" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1705gitlab # [ 220.725186] gitlab-workhorse[825]: time="2026-11-12T09:22:55Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671706gitlab # [ 220.729029] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:55 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 811707gitlab # [ 220.730939] gitaly[899]: time="2026-11-12T09:22:55.734Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M77H2F14AVD2DMQY0ZDNJCTE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:55.699" grpc.time_ms=34.158 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1708gitlab # [ 220.836931] gitaly[899]: time="2026-11-12T09:22:55.870Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M77H2F5M11P5P5PK1YTC9JD0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:55.858" grpc.time_ms=12.64 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1709gitlab # [ 220.838511] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:55 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 761710gitlab # [ 220.889976] gitaly[899]: time="2026-11-12T09:22:55.924Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M77H2F834KH8B82DY3AE88KW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:55.917" pid=899 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1711gitlab # [ 220.890838] gitaly[899]: time="2026-11-12T09:22:55.924Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H2F834KH8B82DY3AE88KW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:55.917" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1712gitlab # [ 220.891513] gitaly[899]: time="2026-11-12T09:22:55.924Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H2F834KH8B82DY3AE88KW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:22:55.917" grpc.time_ms=6.766 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1713gitlab # [ 220.924540] gitlab-workhorse[825]: time="2026-11-12T09:22:55Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671714gitlab # [ 220.925229] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:22:55 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 821715gitlab # [ 221.066394] sidekiq-cluster[2534]: 2026-11-12T09:22:56.099Z pid=2534 tid=xu6 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2534, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>447758336}1716gitlab # [ 221.250292] sidekiq-cluster[2534]: W, [2026-11-12T09:22:56.284310 #2534] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1717gitlab # [ 221.290986] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:22:56.324Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1718gitlab # [ 223.286429] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:22:58.320Z","message":"GitLab reliable fetch activated!"}1719gitlab # [ 223.288066] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:22:58.321Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1720gitlab # [ 244.183863] postgres[3221]: [3221] ERROR: relation "table_without_models" does not exist1721gitlab # [ 244.186284] postgres[3221]: [3221] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1722gitlab # [ 246.606828] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:21.637Z","message":"Booted Rails 7.2.3.1 application in production environment"}1723gitlab # [ 246.608875] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:21.638Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1724gitlab # [ 246.610264] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:21.638Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1725gitlab # [ 246.614170] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:21.638Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1726gitlab # [ 246.615248] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:21.640Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1727gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 43.67 seconds)1728gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1729gitlab # [ 247.310489] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:22 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 441730gitlab # [ 247.405106] gitaly[899]: time="2026-11-12T09:23:22.438Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M77H3939AX6GQ2DCFKH5XYWX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:22.424" grpc.time_ms=13.647 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1731gitlab # [ 247.434265] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:22 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 204 "" "git/2.55.0" 931732gitlab # Enumerating objects: 5, done.1733gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1734gitlab # Delta compression using up to 4 threads1735gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1736gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 91.00 KiB/s, done.1737gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1738gitlab # [ 247.656322] gitaly[899]: time="2026-11-12T09:23:22.690Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=327 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-12T09:23:52.277" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-12T09:23:22.678" grpc.time_ms=11.513 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1739gitlab # [ 247.686603] gitaly[899]: time="2026-11-12T09:23:22.700Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M77H3980EJG8C63ZS7A961EP duration_ms=68ns method=POST pid=899 status=200 url="http://unix/api/v4/internal/allowed"1740gitlab # [ 247.691750] gitaly[899]: time="2026-11-12T09:23:22.720Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M77H3980EJG8C63ZS7A961EP duration_ms=19ns method=POST pid=899 status=200 url="http://unix/api/v4/internal/pre_receive"1741gitlab # [ 247.696983] gitaly[899]: time="2026-11-12T09:23:22.721Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:23:22.630" grpc.time_ms=90.912 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1742gitlab # [ 247.736927] gitaly[899]: time="2026-11-12T09:23:22.771Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:23:22.770" grpc.time_ms=0.434 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1743gitlab # [ 247.770719] gitaly[899]: time="2026-11-12T09:23:22.804Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:23:22.804" grpc.time_ms=0.408 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1744gitlab # [ 247.815072] gitaly[899]: time="2026-11-12T09:23:22.849Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:23:22.848" grpc.time_ms=0.345 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1745gitlab # [ 247.900579] gitaly[899]: time="2026-11-12T09:23:22.934Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M77H3980EJG8C63ZS7A961EP duration_ms=53ns method=POST pid=899 status=200 url="http://unix/api/v4/internal/post_receive"1746gitlab # [ 247.901833] gitaly[899]: time="2026-11-12T09:23:22.935Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:23:22.880" grpc.time_ms=54.98 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1747gitlab # [ 247.943953] gitaly[899]: time="2026-11-12T09:23:22.978Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP diskcache=adb48314-b92a-4da3-b9cd-68c675102fe7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:22.554" pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1748gitlab # [ 247.945162] gitaly[899]: time="2026-11-12T09:23:22.978Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=191 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13447 command.oublock=32 command.real_time_ms=412 command.spawn_token_fork_ms=0 command.system_time_ms=118 command.user_time_ms=73 component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:22.554" grpc.time_ms=423.28 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1749gitlab # [ 247.946577] gitlab-workhorse[825]: time="2026-11-12T09:23:22Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521750gitlab # [ 247.947415] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:22 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 4821751gitlab # To http://gitlab/root/test-flake.git1752gitlab # 85399d3..e262995 master -> master1753gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.78 seconds)1754gitlab: must succeed: git -C /tmp/test-flake rev-parse master1755gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1756gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7'1757gitlab # [ 248.049872] nixbot[2660]: {"ts": "2026-11-12T09:23:23.083Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:39144 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 HTTP/1.1\" 200"}1758gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', in 0.07 seconds)1759gitlab # [ 248.142564] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:23.175Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"76e182b1ce1a05225c03a7e0","created_at":"2026-11-12T09:22:27.810Z","trace_propagation_headers":{"sentry-trace":"aec2965c632e486687b8ceb9e3c3c38e-e25b1c18801b434d","baggage":"sentry-trace_id=aec2965c632e486687b8ceb9e3c3c38e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1K2D9EFHSCYHGSM3HSWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:27.832Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-76e182b1ce1a05225c03a7e0: start","job_status":"start","queue_duration_s":55.342682,"scheduling_latency_s":55.342737}1760gitlab # [ 248.329855] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:23.363Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"76e182b1ce1a05225c03a7e0","created_at":"2026-11-12T09:22:27.810Z","trace_propagation_headers":{"sentry-trace":"aec2965c632e486687b8ceb9e3c3c38e-e25b1c18801b434d","baggage":"sentry-trace_id=aec2965c632e486687b8ceb9e3c3c38e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1K2D9EFHSCYHGSM3HSWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:27.832Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-76e182b1ce1a05225c03a7e0: done: 0.188101 sec","job_status":"done","queue_duration_s":55.342682,"scheduling_latency_s":55.342737,"redis_calls":23,"redis_duration_s":0.009364,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.005536,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002003,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001825,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.096532,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.097498984,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.188101,"completed_at":"2026-11-12T09:23:23.363Z","load_balancing_strategy":"primary","db_duration_s":0.057077,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1761gitlab # [ 248.406090] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:23.439Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"d361d09d3b0bec63f1132be6","created_at":"2026-11-12T09:22:27.841Z","trace_propagation_headers":{"sentry-trace":"aec2965c632e486687b8ceb9e3c3c38e-e25b1c18801b434d","baggage":"sentry-trace_id=aec2965c632e486687b8ceb9e3c3c38e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1K2D9EFHSCYHGSM3HSWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:27.846Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-d361d09d3b0bec63f1132be6: start","job_status":"start","queue_duration_s":55.592563,"scheduling_latency_s":55.592618}1762gitlab # [ 248.535588] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:23.569Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"d361d09d3b0bec63f1132be6","created_at":"2026-11-12T09:22:27.841Z","trace_propagation_headers":{"sentry-trace":"aec2965c632e486687b8ceb9e3c3c38e-e25b1c18801b434d","baggage":"sentry-trace_id=aec2965c632e486687b8ceb9e3c3c38e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1K2D9EFHSCYHGSM3HSWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:27.846Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-d361d09d3b0bec63f1132be6: done: 0.129937 sec","job_status":"done","queue_duration_s":55.592563,"scheduling_latency_s":55.592618,"redis_calls":14,"redis_duration_s":0.003423,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000755,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001609,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001059,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.088805141,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.129937,"completed_at":"2026-11-12T09:23:23.569Z","load_balancing_strategy":"primary","db_duration_s":0.026826,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1763gitlab # [ 248.610914] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:23.644Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"1e9b81f8256d7915e6531a64","created_at":"2026-11-12T09:22:28.352Z","trace_propagation_headers":{"sentry-trace":"aec2965c632e486687b8ceb9e3c3c38e-e25b1c18801b434d","baggage":"sentry-trace_id=aec2965c632e486687b8ceb9e3c3c38e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1K2D9EFHSCYHGSM3HSWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:28.358Z","job_size_bytes":5,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-1e9b81f8256d7915e6531a64: start","job_status":"start","queue_duration_s":55.285588,"scheduling_latency_s":55.285644}1764gitlab # [ 248.790381] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:23.823Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"1e9b81f8256d7915e6531a64","created_at":"2026-11-12T09:22:28.352Z","trace_propagation_headers":{"sentry-trace":"aec2965c632e486687b8ceb9e3c3c38e-e25b1c18801b434d","baggage":"sentry-trace_id=aec2965c632e486687b8ceb9e3c3c38e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1K2D9EFHSCYHGSM3HSWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:28.358Z","job_size_bytes":5,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-1e9b81f8256d7915e6531a64: done: 0.180013 sec","job_status":"done","queue_duration_s":55.285588,"scheduling_latency_s":55.285644,"redis_calls":12,"redis_duration_s":0.00307,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000794,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001803,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000473,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.126027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000310375,"duration_s":0.180013,"completed_at":"2026-11-12T09:23:23.823Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028808,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1765gitlab # [ 248.870702] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:23.904Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"a89729d1d15d8b11b7a75468","created_at":"2026-11-12T09:22:28.381Z","trace_propagation_headers":{"sentry-trace":"aec2965c632e486687b8ceb9e3c3c38e-e25b1c18801b434d","baggage":"sentry-trace_id=aec2965c632e486687b8ceb9e3c3c38e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1K2D9EFHSCYHGSM3HSWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:28.385Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-a89729d1d15d8b11b7a75468: start","job_status":"start","queue_duration_s":55.519062,"scheduling_latency_s":55.519097}1766gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7'1767gitlab # [ 249.035621] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:24.069Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"a89729d1d15d8b11b7a75468","created_at":"2026-11-12T09:22:28.381Z","trace_propagation_headers":{"sentry-trace":"aec2965c632e486687b8ceb9e3c3c38e-e25b1c18801b434d","baggage":"sentry-trace_id=aec2965c632e486687b8ceb9e3c3c38e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1K2D9EFHSCYHGSM3HSWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:28.385Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-a89729d1d15d8b11b7a75468: done: 0.165101 sec","job_status":"done","queue_duration_s":55.519062,"scheduling_latency_s":55.519097,"redis_calls":11,"redis_duration_s":0.0027890000000000002,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000692,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001582,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000515,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.13,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.13,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.091367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000301993,"duration_s":0.165101,"completed_at":"2026-11-12T09:23:24.069Z","load_balancing_strategy":"primary","db_duration_s":0.071504,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1768gitlab # [ 249.038853] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:24.070Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"ea908a56db46fc467101aae3","created_at":"2026-11-12T09:22:28.387Z","trace_propagation_headers":{"sentry-trace":"aec2965c632e486687b8ceb9e3c3c38e-e25b1c18801b434d","baggage":"sentry-trace_id=aec2965c632e486687b8ceb9e3c3c38e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1K2D9EFHSCYHGSM3HSWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:28.391Z","job_size_bytes":89,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-ea908a56db46fc467101aae3: start","job_status":"start","queue_duration_s":55.679571,"scheduling_latency_s":55.679586}1769gitlab # [ 249.131457] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:24.105Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"ea908a56db46fc467101aae3","created_at":"2026-11-12T09:22:28.387Z","trace_propagation_headers":{"sentry-trace":"aec2965c632e486687b8ceb9e3c3c38e-e25b1c18801b434d","baggage":"sentry-trace_id=aec2965c632e486687b8ceb9e3c3c38e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1K2D9EFHSCYHGSM3HSWK","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:28.391Z","job_size_bytes":89,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-ea908a56db46fc467101aae3: done: 0.03493 sec","job_status":"done","queue_duration_s":55.679571,"scheduling_latency_s":55.679586,"redis_calls":11,"redis_duration_s":0.00265,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000747,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00148,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000423,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000310654,"duration_s":0.03493,"completed_at":"2026-11-12T09:23:24.105Z","load_balancing_strategy":"primary","db_duration_s":0.004971,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1770gitlab # [ 249.135287] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:24.107Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 85399d301bf194486bc26d409ef791d202275c25 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"40ceb53a846fb48d101c18a4","created_at":"2026-11-12T09:22:35.769Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-c0dd992336b34001","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:35.790Z","job_size_bytes":185,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-40ceb53a846fb48d101c18a4: start","job_status":"start","queue_duration_s":48.316765,"scheduling_latency_s":48.316778}1771gitlab # [ 249.152513] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:24.119Z","message":"changes: \"0000000000000000000000000000000000000000 85399d301bf194486bc26d409ef791d202275c25 refs/heads/master\\n\""}1772gitlab # [ 249.154666] gitaly[899]: time="2026-11-12T09:23:24.157Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:34.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:24.144" grpc.time_ms=12.219 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1773gitlab # [ 249.204226] gitaly[899]: time="2026-11-12T09:23:24.172Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-12T09:23:34.020" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:23:24.161" grpc.time_ms=10.673 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1774gitlab # [ 249.205112] gitaly[899]: time="2026-11-12T09:23:24.214Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-12T09:23:34.063" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:24.204" grpc.time_ms=10.385 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1775gitlab # [ 249.214947] gitaly[899]: time="2026-11-12T09:23:24.249Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:23:24.248" grpc.time_ms=0.416 pid=899 span.kind=server system=grpc1776gitlab # [ 249.237754] gitaly[899]: time="2026-11-12T09:23:24.271Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:23:24.270" grpc.time_ms=0.86 pid=899 span.kind=server system=grpc1777gitlab # [ 249.250154] gitaly[899]: time="2026-11-12T09:23:24.284Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC diskcache=5daebddd-35e1-4ae8-ab7e-842cb14c668d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-12T09:23:34.081" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:23:24.222" pid=899 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1778gitlab # [ 249.251056] gitaly[899]: time="2026-11-12T09:23:24.284Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=60 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5109 command.oublock=8 command.real_time_ms=52 command.spawn_token_fork_ms=0 command.system_time_ms=34 command.user_time_ms=26 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-12T09:23:34.081" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:23:24.222" grpc.time_ms=62.069 pid=899 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1779gitlab # [ 249.262025] gitaly[899]: time="2026-11-12T09:23:24.296Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-12T09:23:34.155" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:23:24.295" grpc.time_ms=0.147 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1780gitlab # [ 249.274211] gitaly[899]: time="2026-11-12T09:23:24.308Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=941 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-12T09:23:34.159" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:24.300" grpc.time_ms=8.073 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1781gitlab # [ 249.309723] nixbot[2660]: {"ts": "2026-11-12T09:23:24.343Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:39150 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 HTTP/1.1\" 200"}1782gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', in 0.26 seconds)1783gitlab # [ 249.335989] gitaly[899]: time="2026-11-12T09:23:24.370Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=338 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-11-12T09:23:54.065" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:24.365" grpc.time_ms=4.699 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1784gitlab # [ 249.353857] gitaly[899]: time="2026-11-12T09:23:24.388Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-12T09:23:34.247" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:23:24.387" grpc.time_ms=0.197 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1785gitlab # [ 249.361608] gitaly[899]: time="2026-11-12T09:23:24.395Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-12T09:23:34.251" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-11-12T09:23:24.391" grpc.time_ms=3.928 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1786gitlab # [ 249.399586] gitaly[899]: time="2026-11-12T09:23:24.433Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:34.286" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:24.426" grpc.time_ms=6.509 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1787gitlab # [ 249.411234] gitaly[899]: time="2026-11-12T09:23:24.445Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:34.297" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:24.437" grpc.time_ms=7.596 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1788gitlab # [ 249.431765] gitaly[899]: time="2026-11-12T09:23:24.465Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-12T09:23:54.153" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:24.453" grpc.time_ms=12.37 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1789gitlab # [ 249.793056] gitaly[899]: time="2026-11-12T09:23:24.827Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-12T09:23:34.004" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:24.825" grpc.time_ms=1.84 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1790gitlab # [ 249.825608] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:24.858Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 85399d301bf194486bc26d409ef791d202275c25 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"40ceb53a846fb48d101c18a4","created_at":"2026-11-12T09:22:35.769Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-c0dd992336b34001","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:22:35.790Z","job_size_bytes":185,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-40ceb53a846fb48d101c18a4: done: 0.751768 sec","job_status":"done","queue_duration_s":48.316765,"scheduling_latency_s":48.316778,"gitaly_calls":13,"gitaly_duration_s":0.191717,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.038539000000000004,"redis_read_bytes":2103,"redis_write_bytes":21748,"redis_cache_calls":15,"redis_cache_duration_s":0.002706,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.006329,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001494,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7836,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004786,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.012157,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.011067,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.163,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.179,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.466124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000477155,"exclusive_lock_hold_duration_s":0.030957844999999998,"duration_s":0.751768,"completed_at":"2026-11-12T09:23:24.858Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.076926,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1791gitlab # [ 249.893162] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:24.926Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"9fcb2402a6d245e83e81f1e2","created_at":"2026-11-12T09:22:40.730Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-82ef7da103994da3","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:40.736Z","job_size_bytes":65,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-9fcb2402a6d245e83e81f1e2: start","job_status":"start","queue_duration_s":44.189881,"scheduling_latency_s":44.189924}1792gitlab # [ 250.023245] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:25.056Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"9fcb2402a6d245e83e81f1e2","created_at":"2026-11-12T09:22:40.730Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-82ef7da103994da3","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:40.736Z","job_size_bytes":65,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-9fcb2402a6d245e83e81f1e2: done: 0.130035 sec","job_status":"done","queue_duration_s":44.189881,"scheduling_latency_s":44.189924,"redis_calls":16,"redis_duration_s":0.0046689999999999995,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002436,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001635,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000598,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.078086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000436927,"duration_s":0.130035,"completed_at":"2026-11-12T09:23:25.056Z","load_balancing_strategy":"primary","db_duration_s":0.025808,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1793gitlab # [ 250.086051] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:25.119Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"895f37895e820330ea0be72a","created_at":"2026-11-12T09:22:40.737Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-82ef7da103994da3","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:40.740Z","job_size_bytes":65,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-895f37895e820330ea0be72a: start","job_status":"start","queue_duration_s":44.379226,"scheduling_latency_s":44.379287}1794gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7'1795gitlab # [ 250.270553] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:25.303Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"895f37895e820330ea0be72a","created_at":"2026-11-12T09:22:40.737Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-82ef7da103994da3","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:40.740Z","job_size_bytes":65,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-895f37895e820330ea0be72a: done: 0.184205 sec","job_status":"done","queue_duration_s":44.379226,"scheduling_latency_s":44.379287,"redis_calls":21,"redis_duration_s":0.006215,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002576,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001838,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001801,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.136722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034781,"duration_s":0.184205,"completed_at":"2026-11-12T09:23:25.303Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033807,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1796gitlab # [ 250.274469] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:25.305Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"595f31df6b9d8425b67b5f89","created_at":"2026-11-12T09:22:40.938Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-82ef7da103994da3","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:40.943Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-595f31df6b9d8425b67b5f89: start","job_status":"start","queue_duration_s":44.361305,"scheduling_latency_s":44.361319}1797gitlab # [ 250.395287] nixbot[2660]: {"ts": "2026-11-12T09:23:25.428Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:39166 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 HTTP/1.1\" 200"}1798gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', in 0.08 seconds)1799gitlab # [ 250.516853] gitaly[899]: time="2026-11-12T09:23:25.550Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:23:25.549" grpc.time_ms=0.555 pid=899 span.kind=server system=grpc1800gitlab # [ 250.557333] gitaly[899]: time="2026-11-12T09:23:25.591Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:23:25.590" grpc.time_ms=0.553 pid=899 span.kind=server system=grpc1801gitlab # [ 250.603517] gitaly[899]: time="2026-11-12T09:23:25.637Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:23:25.636" grpc.time_ms=0.352 pid=899 span.kind=server system=grpc1802gitlab # [ 250.635264] gitaly[899]: time="2026-11-12T09:23:25.669Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP diskcache=bd7653ae-be0b-4570-8edf-ac4d22661f4b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-12T09:23:55.222" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:25.522" pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1803gitlab # [ 250.636281] gitaly[899]: time="2026-11-12T09:23:25.669Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=136 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7610 command.oublock=0 command.real_time_ms=134 command.spawn_token_fork_ms=0 command.system_time_ms=81 command.user_time_ms=55 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-12T09:23:55.222" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:25.522" grpc.time_ms=146.54 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1804gitlab # [ 250.784163] gitaly[899]: time="2026-11-12T09:23:25.818Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-12T09:23:55.516" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:25.816" grpc.time_ms=1.286 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1805gitlab # [ 250.838141] gitaly[899]: time="2026-11-12T09:23:25.872Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1ZZMG0R1FM5K6ASRM7SP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:23:55.569" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:25.870" grpc.time_ms=1.13 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1806gitlab # [ 250.857162] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:25.890Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"595f31df6b9d8425b67b5f89","created_at":"2026-11-12T09:22:40.938Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-82ef7da103994da3","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:40.943Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2534,"job_status":"rescheduled","message":"PipelineProcessWorker JID-595f31df6b9d8425b67b5f89: rescheduled"}1807gitlab # [ 250.871982] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:25.905Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"595f31df6b9d8425b67b5f89","created_at":"2026-11-12T09:22:40.938Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-82ef7da103994da3","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-12T09:22:40.943Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-595f31df6b9d8425b67b5f89: done: 0.600445 sec","job_status":"done","queue_duration_s":44.361305,"scheduling_latency_s":44.361319,"gitaly_calls":3,"gitaly_duration_s":0.158976,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.035355,"redis_read_bytes":460,"redis_write_bytes":29295,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00087,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002292,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.013711,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.006155,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15582,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.007893,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000539,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000672,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002353,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.148,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.016,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.019,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.122,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.258811,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.4324125,"duration_s":0.600445,"completed_at":"2026-11-12T09:23:25.905Z","load_balancing_strategy":"primary","db_duration_s":0.159033,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1808gitlab # [ 250.937641] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:25.907Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","85399d301bf194486bc26d409ef791d202275c25 e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"65c95d04307ee60d58f40ff4","created_at":"2026-11-12T09:23:22.913Z","trace_propagation_headers":{"sentry-trace":"009f533470024edbb821b9e5537f7512-bfc28f4f11bc4e62","baggage":"sentry-trace_id=009f533470024edbb821b9e5537f7512,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H3980EJG8C63ZS7A961EP","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:23:22.916Z","job_size_bytes":185,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-65c95d04307ee60d58f40ff4: start","job_status":"start","queue_duration_s":2.991325,"scheduling_latency_s":2.991359}1809gitlab # [ 250.938732] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:25.922Z","message":"changes: \"85399d301bf194486bc26d409ef791d202275c25 e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 refs/heads/master\\n\""}1810gitlab # [ 250.939733] gitaly[899]: time="2026-11-12T09:23:25.952Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:35.122" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:25.942" grpc.time_ms=9.948 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1811gitlab # [ 250.954650] gitaly[899]: time="2026-11-12T09:23:25.988Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-12T09:23:35.159" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:23:25.980" grpc.time_ms=8.46 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1812gitlab # [ 250.984060] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:26.017Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"e5f03d42407e0c25fbc56566","created_at":"2026-11-12T09:23:26.015Z","trace_propagation_headers":{"sentry-trace":"009f533470024edbb821b9e5537f7512-c642a8b4a96a4a69","baggage":"sentry-trace_id=009f533470024edbb821b9e5537f7512,sentry-release=Unknown"},"correlation_id":"01M77H3980EJG8C63ZS7A961EP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"292ed15e3b192b2e0d1b2954","job_size_bytes":46,"pid":2534,"job_status":"deduplicated","message":"ProjectCacheWorker JID-e5f03d42407e0c25fbc56566: deduplicated: until executing","deduplication.type":"until executing"}1813gitlab # [ 251.006363] gitaly[899]: time="2026-11-12T09:23:26.040Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-12T09:23:36.007" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:26.028" grpc.time_ms=11.918 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1814gitlab # [ 251.024368] gitaly[899]: time="2026-11-12T09:23:26.058Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-11-12T09:23:56.046" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:26.047" grpc.time_ms=11.224 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1815gitlab # [ 251.044256] gitaly[899]: time="2026-11-12T09:23:26.078Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-12T09:23:36.051" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-11-12T09:23:26.072" grpc.time_ms=6.092 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1816gitlab # [ 251.067393] gitaly[899]: time="2026-11-12T09:23:26.101Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:36.070" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:26.091" grpc.time_ms=10.333 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1817gitlab # [ 251.160322] gitaly[899]: time="2026-11-12T09:23:26.194Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-12T09:23:36.052" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:26.192" grpc.time_ms=1.503 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1818gitlab # [ 251.192357] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:26.225Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","85399d301bf194486bc26d409ef791d202275c25 e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"65c95d04307ee60d58f40ff4","created_at":"2026-11-12T09:23:22.913Z","trace_propagation_headers":{"sentry-trace":"009f533470024edbb821b9e5537f7512-bfc28f4f11bc4e62","baggage":"sentry-trace_id=009f533470024edbb821b9e5537f7512,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H3980EJG8C63ZS7A961EP","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:23:22.916Z","job_size_bytes":185,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-65c95d04307ee60d58f40ff4: done: 0.317821 sec","job_status":"done","queue_duration_s":2.991325,"scheduling_latency_s":2.991359,"gitaly_calls":7,"gitaly_duration_s":0.128336,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.023134000000000002,"redis_read_bytes":501,"redis_write_bytes":11944,"redis_cache_calls":9,"redis_cache_duration_s":0.002562,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001118,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5721,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003458,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.007973,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.008023,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.025,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.025,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.15334,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000769931,"exclusive_lock_hold_duration_s":0.041199929,"duration_s":0.317821,"completed_at":"2026-11-12T09:23:26.225Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026211,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1819gitlab # [ 251.202452] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:26.227Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"292ed15e3b192b2e0d1b2954","created_at":"2026-11-12T09:23:24.174Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-a67565109b814fe3","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:24.188Z","job_size_bytes":46,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-292ed15e3b192b2e0d1b2954: start","job_status":"start","queue_duration_s":2.039864,"scheduling_latency_s":2.039901}1820gitlab # [ 251.254770] gitaly[899]: time="2026-11-12T09:23:26.268Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-11-12T15:23:26.265" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:23:26.266" grpc.time_ms=2.26 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1821gitlab # [ 251.255720] gitaly[899]: time="2026-11-12T09:23:26.285Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-12T09:23:56.273" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:26.274" grpc.time_ms=10.959 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1822gitlab # [ 251.270558] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:26.303Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"292ed15e3b192b2e0d1b2954","created_at":"2026-11-12T09:23:24.174Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-a67565109b814fe3","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:24.188Z","job_size_bytes":46,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-292ed15e3b192b2e0d1b2954: done: 0.075865 sec","job_status":"done","queue_duration_s":2.039864,"scheduling_latency_s":2.039901,"gitaly_calls":2,"gitaly_duration_s":0.0169,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.004883,"redis_read_bytes":69,"redis_write_bytes":4888,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001189,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000407,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001097,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.00164,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00055,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208406,"duration_s":0.075865,"completed_at":"2026-11-12T09:23:26.303Z","load_balancing_strategy":"primary","db_duration_s":0.008458,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1823gitlab # [ 251.329228] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:26.305Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"7cffca5dc19658f21aa8af3d","created_at":"2026-11-12T09:23:24.197Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-a67565109b814fe3","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:24.200Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-7cffca5dc19658f21aa8af3d: start","job_status":"start","queue_duration_s":2.104136,"scheduling_latency_s":2.104151}1824gitlab # [ 251.330052] gitaly[899]: time="2026-11-12T09:23:26.343Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC diskcache=11a6c822-6b14-4efa-be59-92e9d1ddb9b2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-12T15:23:26.319" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:26.320" pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1825gitlab # [ 251.330793] gitaly[899]: time="2026-11-12T09:23:26.344Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1288 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-12T15:23:26.319" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:26.320" grpc.time_ms=23.815 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1826gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7'1827gitlab # [ 251.334331] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:26.368Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"7cffca5dc19658f21aa8af3d","created_at":"2026-11-12T09:23:24.197Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-a67565109b814fe3","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:24.200Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-7cffca5dc19658f21aa8af3d: done: 0.063123 sec","job_status":"done","queue_duration_s":2.104136,"scheduling_latency_s":2.104151,"gitaly_calls":1,"gitaly_duration_s":0.02566,"redis_calls":15,"redis_duration_s":0.002022,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000752,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000365,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000652,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.047728134,"duration_s":0.063123,"completed_at":"2026-11-12T09:23:26.368Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009124,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1828gitlab # [ 251.398386] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:26.369Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"09bd134e0ca96291cecb98d0","created_at":"2026-11-12T09:23:24.292Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-a67565109b814fe3","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:24.293Z","job_size_bytes":23,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-09bd134e0ca96291cecb98d0: start","job_status":"start","queue_duration_s":2.075406,"scheduling_latency_s":2.075421}1829gitlab # [ 251.399021] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:26.398Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"09bd134e0ca96291cecb98d0","created_at":"2026-11-12T09:23:24.292Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-a67565109b814fe3","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:24.293Z","job_size_bytes":23,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-09bd134e0ca96291cecb98d0: done: 0.02902 sec","job_status":"done","queue_duration_s":2.075406,"scheduling_latency_s":2.075421,"gitaly_calls":1,"gitaly_duration_s":0.007093,"redis_calls":16,"redis_duration_s":0.002175,"redis_read_bytes":52,"redis_write_bytes":2045,"redis_queues_calls":1,"redis_queues_duration_s":0.000208,"redis_queues_read_bytes":1,"redis_queues_write_bytes":976,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000617,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.000926,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000424,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02902,"completed_at":"2026-11-12T09:23:26.398Z","load_balancing_strategy":"primary","db_duration_s":0.00166,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1830gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', in 0.07 seconds)1831gitlab # [ 251.444323] gitaly[899]: time="2026-11-12T09:23:26.392Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-12T09:23:56.086" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:26.386" grpc.time_ms=5.469 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1832gitlab # [ 251.445075] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:26.399Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"eae7a714f27dff5997df46c8","created_at":"2026-11-12T09:23:24.667Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-a67565109b814fe3","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a7ce35c7d456a2470a9b1e427b6ef95ecc795f081b9a3e130eca8e64695d9c11","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:24.672Z","job_size_bytes":467,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-eae7a714f27dff5997df46c8: start","job_status":"start","queue_duration_s":1.727289,"scheduling_latency_s":1.727305}1833gitlab # [ 251.452694] gitaly[899]: time="2026-11-12T09:23:26.416Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:23:56.115" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:26.415" grpc.time_ms=0.499 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1834gitlab # [ 251.492354] gitaly[899]: time="2026-11-12T09:23:26.423Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-12T09:23:36.007" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:26.417" grpc.time_ms=5.469 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1835gitlab # [ 251.493287] nixbot[2660]: {"ts": "2026-11-12T09:23:26.502Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:39180 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 HTTP/1.1\" 200"}1836gitlab # [ 251.493770] gitaly[899]: time="2026-11-12T09:23:26.500Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-12T09:23:56.498" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:26.499" grpc.time_ms=0.846 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1837gitlab # [ 251.494457] gitaly[899]: time="2026-11-12T09:23:26.505Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=61 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-12T09:23:56.502" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:26.503" grpc.time_ms=2.408 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1838gitlab # [ 251.529456] gitaly[899]: time="2026-11-12T09:23:26.563Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:36.146" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:26.557" grpc.time_ms=5.941 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1839gitlab # [ 251.914909] gitaly[899]: time="2026-11-12T09:23:26.948Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-12T09:23:56.030" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:26.930" grpc.time_ms=18.087 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1840gitlab # [ 251.947745] gitaly[899]: time="2026-11-12T09:23:26.968Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H1V0BNZ4JGW1XVJEH6DKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-12T09:23:56.050" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:26.951" grpc.time_ms=16.505 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1841gitlab # [ 251.972344] sidekiq-cluster[2534]: {"severity":"WARN","time":"2026-11-12T09:23:26.990Z","class":"CreatePipelineWorker","correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"85399d301bf194486bc26d409ef791d202275c25","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"85399d301bf194486bc26d409ef791d202275c25","ref":"refs/heads/master"},"retry":0}1842gitlab # [ 251.980239] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:26.995Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"eae7a714f27dff5997df46c8","created_at":"2026-11-12T09:23:24.667Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-a67565109b814fe3","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a7ce35c7d456a2470a9b1e427b6ef95ecc795f081b9a3e130eca8e64695d9c11","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:24.672Z","job_size_bytes":467,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-eae7a714f27dff5997df46c8: done: 0.596334 sec","job_status":"done","queue_duration_s":1.727289,"scheduling_latency_s":1.727305,"gitaly_calls":7,"gitaly_duration_s":0.063205,"redis_calls":51,"redis_duration_s":0.008302,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000914,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.003681,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001057,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001761,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000889,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.486369,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000284115,"duration_s":0.596334,"completed_at":"2026-11-12T09:23:26.995Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035869,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1843gitlab # [ 251.985273] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:26.997Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"48b1b18a-279a-4996-b03b-3c1150d1a6bd\"}"],"class":"WebHookWorker","jid":"b42ad2aaf17d378fd1746b9c","created_at":"2026-11-12T09:23:24.792Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-a67565109b814fe3","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc4afddcda2487feceec312335d567bd4e1e6b59c45ab9ea430fa2154e45c1a8","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:24.799Z","job_size_bytes":1751,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-b42ad2aaf17d378fd1746b9c: start","job_status":"start","queue_duration_s":2.197616,"scheduling_latency_s":2.197632}1844gitlab # [ 252.148355] nixbot[2660]: {"ts": "2026-11-12T09:23:27.182Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "3babd0ae-e34c-44e6-a653-f0bc539a9d2f", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "85399d301bf194486bc26d409ef791d202275c25", "pr": null}1845gitlab # [ 252.153641] nixbot[2660]: {"ts": "2026-11-12T09:23:27.187Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:39192 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}1846gitlab # [ 252.178819] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.212Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"48b1b18a-279a-4996-b03b-3c1150d1a6bd\"}"],"class":"WebHookWorker","jid":"b42ad2aaf17d378fd1746b9c","created_at":"2026-11-12T09:23:24.792Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-a67565109b814fe3","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc4afddcda2487feceec312335d567bd4e1e6b59c45ab9ea430fa2154e45c1a8","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:24.799Z","job_size_bytes":1751,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-b42ad2aaf17d378fd1746b9c: done: 0.215052 sec","job_status":"done","queue_duration_s":2.197616,"scheduling_latency_s":2.197632,"redis_calls":19,"redis_duration_s":0.004836,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001176,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000608,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002092,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.064,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.064,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.06638385599998742,"cpu_s":0.070472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000582476,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.215052,"completed_at":"2026-11-12T09:23:27.212Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.063491,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1847gitlab # [ 252.251387] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.285Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3b72893b5fef684f1a280b68","created_at":"2026-11-12T09:23:25.504Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.511Z","job_size_bytes":23,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3b72893b5fef684f1a280b68: start","job_status":"start","queue_duration_s":1.773636,"scheduling_latency_s":1.773673}1848gitlab # [ 252.346520] gitaly[899]: time="2026-11-12T09:23:27.380Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M77H3DVMMH0TY2MWBH6XBGQJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:27.370" grpc.time_ms=9.515 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1849gitlab # [ 252.366129] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.399Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3b72893b5fef684f1a280b68","created_at":"2026-11-12T09:23:25.504Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.511Z","job_size_bytes":23,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3b72893b5fef684f1a280b68: done: 0.114474 sec","job_status":"done","queue_duration_s":1.773636,"scheduling_latency_s":1.773673,"redis_calls":10,"redis_duration_s":0.0030470000000000002,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000875,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001636,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000536,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.071858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000272661,"duration_s":0.114474,"completed_at":"2026-11-12T09:23:27.399Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019266,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1850gitlab # [ 252.371927] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.405Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d90e066ce81530bc9bc4b87b","created_at":"2026-11-12T09:23:25.677Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.684Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d90e066ce81530bc9bc4b87b: start","job_status":"start","queue_duration_s":1.720855,"scheduling_latency_s":1.720871}1851gitlab # [ 252.398570] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.432Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d90e066ce81530bc9bc4b87b","created_at":"2026-11-12T09:23:25.677Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.684Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d90e066ce81530bc9bc4b87b: done: 0.026459 sec","job_status":"done","queue_duration_s":1.720855,"scheduling_latency_s":1.720871,"redis_calls":10,"redis_duration_s":0.003287,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000453,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002475,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000359,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000243327,"duration_s":0.026459,"completed_at":"2026-11-12T09:23:27.432Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005793,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1852gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7'1853gitlab # [ 252.439231] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.434Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"bd1662797e4c8aa7bc145f98","created_at":"2026-11-12T09:23:25.701Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.708Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-bd1662797e4c8aa7bc145f98: start","job_status":"start","queue_duration_s":1.725572,"scheduling_latency_s":1.725602}1854gitlab # [ 252.443196] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.459Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"bd1662797e4c8aa7bc145f98","created_at":"2026-11-12T09:23:25.701Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.708Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-bd1662797e4c8aa7bc145f98: done: 0.025683 sec","job_status":"done","queue_duration_s":1.725572,"scheduling_latency_s":1.725602,"redis_calls":10,"redis_duration_s":0.001405,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000293,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000851,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000261,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000311213,"duration_s":0.025683,"completed_at":"2026-11-12T09:23:27.459Z","load_balancing_strategy":"primary","db_duration_s":0.003772,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1855gitlab # [ 252.482380] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.460Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8d1148223b8ff5d419bdd51f","created_at":"2026-11-12T09:23:25.724Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.729Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8d1148223b8ff5d419bdd51f: start","job_status":"start","queue_duration_s":1.730917,"scheduling_latency_s":1.730931}1856gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', in 0.08 seconds)1857gitlab # [ 252.522616] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.486Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8d1148223b8ff5d419bdd51f","created_at":"2026-11-12T09:23:25.724Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.729Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8d1148223b8ff5d419bdd51f: done: 0.025813 sec","job_status":"done","queue_duration_s":1.730917,"scheduling_latency_s":1.730931,"redis_calls":13,"redis_duration_s":0.003882,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00081,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002586,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000244,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000225447,"duration_s":0.025813,"completed_at":"2026-11-12T09:23:27.486Z","load_balancing_strategy":"primary","db_duration_s":0.003991,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1858gitlab # [ 252.562698] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.487Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ebd49ee5a67dca84b64c6795","created_at":"2026-11-12T09:23:25.730Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.737Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ebd49ee5a67dca84b64c6795: start","job_status":"start","queue_duration_s":1.75035,"scheduling_latency_s":1.750364}1859gitlab # [ 252.563489] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.507Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"ebd49ee5a67dca84b64c6795","created_at":"2026-11-12T09:23:25.730Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.737Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-ebd49ee5a67dca84b64c6795: done: 0.01964 sec","job_status":"done","queue_duration_s":1.75035,"scheduling_latency_s":1.750364,"redis_calls":10,"redis_duration_s":0.001277,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000787,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000228521,"duration_s":0.01964,"completed_at":"2026-11-12T09:23:27.506Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00263,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1860gitlab # [ 252.589874] nixbot[2660]: {"ts": "2026-11-12T09:23:27.582Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:39200 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 HTTP/1.1\" 200"}1861gitlab # [ 252.591975] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.507Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"86830ba7f9bebabc9efe2ee1","created_at":"2026-11-12T09:23:25.738Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.747Z","job_size_bytes":19,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-86830ba7f9bebabc9efe2ee1: start","job_status":"start","queue_duration_s":1.76019,"scheduling_latency_s":1.760203}1862gitlab # [ 252.645140] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.529Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"86830ba7f9bebabc9efe2ee1","created_at":"2026-11-12T09:23:25.738Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.747Z","job_size_bytes":19,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-86830ba7f9bebabc9efe2ee1: done: 0.021727 sec","job_status":"done","queue_duration_s":1.76019,"scheduling_latency_s":1.760203,"redis_calls":11,"redis_duration_s":0.0028599999999999997,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000247,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001372,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001241,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000508724,"duration_s":0.021727,"completed_at":"2026-11-12T09:23:27.529Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00431,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1863gitlab # [ 252.646939] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.531Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c2f6ed11c356f8f4ac956770","created_at":"2026-11-12T09:23:25.752Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.758Z","job_size_bytes":66,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c2f6ed11c356f8f4ac956770: start","job_status":"start","queue_duration_s":1.77308,"scheduling_latency_s":1.773094}1864gitlab # [ 252.692097] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.579Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c2f6ed11c356f8f4ac956770","created_at":"2026-11-12T09:23:25.752Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.758Z","job_size_bytes":66,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c2f6ed11c356f8f4ac956770: done: 0.047942 sec","job_status":"done","queue_duration_s":1.77308,"scheduling_latency_s":1.773094,"redis_calls":10,"redis_duration_s":0.016161,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002912,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012976,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000273,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000398095,"duration_s":0.047942,"completed_at":"2026-11-12T09:23:27.579Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013714,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1865gitlab # [ 252.702896] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.580Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4a6370cde1a46a9aa57224c1","created_at":"2026-11-12T09:23:25.759Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.766Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4a6370cde1a46a9aa57224c1: start","job_status":"start","queue_duration_s":1.814644,"scheduling_latency_s":1.814657}1866gitlab # [ 252.754585] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.613Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4a6370cde1a46a9aa57224c1","created_at":"2026-11-12T09:23:25.759Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.766Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4a6370cde1a46a9aa57224c1: done: 0.032443 sec","job_status":"done","queue_duration_s":1.814644,"scheduling_latency_s":1.814657,"redis_calls":10,"redis_duration_s":0.002253,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000247,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001297,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000709,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000231314,"duration_s":0.032443,"completed_at":"2026-11-12T09:23:27.613Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006005,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1867gitlab # [ 252.757111] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.614Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"0efc267ebacb5335bad75ad6","created_at":"2026-11-12T09:23:25.776Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.782Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-0efc267ebacb5335bad75ad6: start","job_status":"start","queue_duration_s":1.832273,"scheduling_latency_s":1.832288}1868gitlab # [ 252.795928] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.639Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"0efc267ebacb5335bad75ad6","created_at":"2026-11-12T09:23:25.776Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.782Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-0efc267ebacb5335bad75ad6: done: 0.024766 sec","job_status":"done","queue_duration_s":1.832273,"scheduling_latency_s":1.832288,"redis_calls":10,"redis_duration_s":0.001412,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014089,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000239416,"duration_s":0.024766,"completed_at":"2026-11-12T09:23:27.639Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006162,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1869gitlab # [ 252.842313] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.640Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"6882de1fe41ace04ae3f930d","created_at":"2026-11-12T09:23:25.783Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.787Z","job_size_bytes":6,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-6882de1fe41ace04ae3f930d: start","job_status":"start","queue_duration_s":1.852402,"scheduling_latency_s":1.852415}1870gitlab # [ 252.843051] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.654Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"6882de1fe41ace04ae3f930d","created_at":"2026-11-12T09:23:25.783Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.787Z","job_size_bytes":6,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-6882de1fe41ace04ae3f930d: done: 0.014505 sec","job_status":"done","queue_duration_s":1.852402,"scheduling_latency_s":1.852415,"redis_calls":10,"redis_duration_s":0.001335,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000251,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000298,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008537,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212597,"duration_s":0.014505,"completed_at":"2026-11-12T09:23:27.654Z","load_balancing_strategy":"primary","db_duration_s":0.001572,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1871gitlab # [ 252.890140] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.655Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"0b217682ed552e4cd2929aa9","created_at":"2026-11-12T09:23:25.788Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.793Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-0b217682ed552e4cd2929aa9: start","job_status":"start","queue_duration_s":1.86255,"scheduling_latency_s":1.862563}1872gitlab # [ 252.890697] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.755Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"0b217682ed552e4cd2929aa9","created_at":"2026-11-12T09:23:25.788Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.793Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-0b217682ed552e4cd2929aa9: done: 0.099944 sec","job_status":"done","queue_duration_s":1.86255,"scheduling_latency_s":1.862563,"redis_calls":10,"redis_duration_s":0.001493,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000967,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.087674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000241651,"duration_s":0.099944,"completed_at":"2026-11-12T09:23:27.755Z","load_balancing_strategy":"primary","db_duration_s":0.004472,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1873gitlab # [ 252.917169] gitaly[899]: time="2026-11-12T09:23:27.893Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP diskcache=64ad2898-9181-4240-99d3-fd087a75c17b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-12T15:23:27.830" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:27.831" pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1874gitlab # [ 252.925758] gitaly[899]: time="2026-11-12T09:23:27.894Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=317 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-12T15:23:27.830" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:27.831" grpc.time_ms=62.888 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1875gitlab # [ 252.971409] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.757Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"5669fcac57d8548ea44e62fc","created_at":"2026-11-12T09:23:25.800Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.805Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-5669fcac57d8548ea44e62fc: start","job_status":"start","queue_duration_s":1.951297,"scheduling_latency_s":1.951319}1876gitlab # [ 252.972068] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.801Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"5669fcac57d8548ea44e62fc","created_at":"2026-11-12T09:23:25.800Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.805Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-5669fcac57d8548ea44e62fc: done: 0.044317 sec","job_status":"done","queue_duration_s":1.951297,"scheduling_latency_s":1.951319,"redis_calls":14,"redis_duration_s":0.002565,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000259,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001498,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000808,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000499784,"duration_s":0.044317,"completed_at":"2026-11-12T09:23:27.801Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009087,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1877gitlab # [ 252.991912] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.802Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f7c1a5b3bfed7716c66e5ada","created_at":"2026-11-12T09:23:25.891Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.894Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f7c1a5b3bfed7716c66e5ada: start","job_status":"start","queue_duration_s":1.908079,"scheduling_latency_s":1.908093}1878gitlab # [ 253.005784] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.818Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f7c1a5b3bfed7716c66e5ada","created_at":"2026-11-12T09:23:25.891Z","trace_propagation_headers":{"sentry-trace":"f382027d8f0b444598678990aa234a66-f2c006c8e96f4474","baggage":"sentry-trace_id=f382027d8f0b444598678990aa234a66,sentry-release=Unknown"},"correlation_id":"01M77H1ZZMG0R1FM5K6ASRM7SP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:25.894Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f7c1a5b3bfed7716c66e5ada: done: 0.016227 sec","job_status":"done","queue_duration_s":1.908079,"scheduling_latency_s":1.908093,"redis_calls":8,"redis_duration_s":0.001182,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016227,"completed_at":"2026-11-12T09:23:27.818Z","load_balancing_strategy":"primary","db_duration_s":0.002197,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1879gitlab # [ 253.050926] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.819Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"cba9346c7d48f2a3bb130948","created_at":"2026-11-12T09:23:26.022Z","trace_propagation_headers":{"sentry-trace":"009f533470024edbb821b9e5537f7512-c642a8b4a96a4a69","baggage":"sentry-trace_id=009f533470024edbb821b9e5537f7512,sentry-release=Unknown"},"correlation_id":"01M77H3980EJG8C63ZS7A961EP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"7cffca5dc19658f21aa8af3d","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:26.024Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-cba9346c7d48f2a3bb130948: start","job_status":"start","queue_duration_s":1.795009,"scheduling_latency_s":1.795023}1880gitlab # [ 253.090581] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.906Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"cba9346c7d48f2a3bb130948","created_at":"2026-11-12T09:23:26.022Z","trace_propagation_headers":{"sentry-trace":"009f533470024edbb821b9e5537f7512-c642a8b4a96a4a69","baggage":"sentry-trace_id=009f533470024edbb821b9e5537f7512,sentry-release=Unknown"},"correlation_id":"01M77H3980EJG8C63ZS7A961EP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"7cffca5dc19658f21aa8af3d","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:26.024Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-cba9346c7d48f2a3bb130948: done: 0.087092 sec","job_status":"done","queue_duration_s":1.795009,"scheduling_latency_s":1.795023,"gitaly_calls":1,"gitaly_duration_s":0.065203,"redis_calls":12,"redis_duration_s":0.001684,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000332,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000702,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.075207044,"duration_s":0.087092,"completed_at":"2026-11-12T09:23:27.906Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002763,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1881gitlab # [ 253.133227] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:27.908Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"482bebb908737aadbe460ce3","created_at":"2026-11-12T09:23:26.152Z","trace_propagation_headers":{"sentry-trace":"009f533470024edbb821b9e5537f7512-c642a8b4a96a4a69","baggage":"sentry-trace_id=009f533470024edbb821b9e5537f7512,sentry-release=Unknown"},"correlation_id":"01M77H3980EJG8C63ZS7A961EP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b89c2c349a4d3c6d2bda7ef12bbc8d520ded6357aa04072972e66dea320947ed","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:26.156Z","job_size_bytes":467,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-482bebb908737aadbe460ce3: start","job_status":"start","queue_duration_s":1.752433,"scheduling_latency_s":1.752448}1882gitlab # [ 253.154089] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:28 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 9661883gitlab # [ 253.154845] gitaly[899]: time="2026-11-12T09:23:28.187Z" level=info msg="finished unary call with code OK" catfile.duration_ms=263 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=263 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:23:57.023" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:27.923" grpc.time_ms=263.517 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1884gitlab # [ 253.175767] gitaly[899]: time="2026-11-12T09:23:28.209Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-12T09:23:38.183" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:28.204" grpc.time_ms=5.622 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1885gitlab # [ 253.199452] gitaly[899]: time="2026-11-12T09:23:28.233Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-12T09:23:58.232" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:28.232" grpc.time_ms=0.712 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1886gitlab # [ 253.218682] gitaly[899]: time="2026-11-12T09:23:28.252Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=59 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-12T09:23:58.249" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:28.249" grpc.time_ms=3.004 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1887gitlab # [ 253.233646] gitaly[899]: time="2026-11-12T09:23:28.266Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M77H3ESXZ8EXQ4XRF1VQTASY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:28.258" pid=899 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1888gitlab # [ 253.234968] gitaly[899]: time="2026-11-12T09:23:28.267Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3ESXZ8EXQ4XRF1VQTASY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:28.258" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1889gitlab # [ 253.235647] gitlab-workhorse[825]: time="2026-11-12T09:23:28Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671890gitlab # [ 253.236234] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:28 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 791891gitlab # [ 253.237574] gitaly[899]: time="2026-11-12T09:23:28.267Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3ESXZ8EXQ4XRF1VQTASY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:28.258" grpc.time_ms=9.465 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1892gitlab # [ 253.285070] gitaly[899]: time="2026-11-12T09:23:28.319Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M77H3EWF26QX4NP1SDAPMC1Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:28.307" pid=899 remote_ip="2001:db8:1::1" request_sha=a48426d5574e68f7c73b16bcfab273df587279ca response_bytes=338 span.kind=server system=grpc1893gitlab # [ 253.285533] gitaly[899]: time="2026-11-12T09:23:28.319Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1193 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3EWF26QX4NP1SDAPMC1Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:28.307" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1894gitlab # [ 253.286320] gitaly[899]: time="2026-11-12T09:23:28.320Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1193 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3EWF26QX4NP1SDAPMC1Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:28.307" grpc.time_ms=12.616 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1895gitlab # [ 253.286997] gitlab-workhorse[825]: time="2026-11-12T09:23:28Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3381896gitlab # [ 253.288934] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:28 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 338 "" "git/2.55.0" 491897gitlab # [ 253.317049] nixbot[2660]: {"ts": "2026-11-12T09:23:28.351Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1898gitlab # [ 253.318382] nixbot[2660]: {"ts": "2026-11-12T09:23:28.352Z", "level": "info", "logger": "nixbot.after_build", "msg": "reused build: replaying effect statuses", "build_id": 1, "sha": "85399d301bf194486bc26d409ef791d202275c25"}1899gitlab # [ 253.348312] gitaly[899]: time="2026-11-12T09:23:28.382Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:38.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:28.377" grpc.time_ms=5.349 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900gitlab # [ 253.372100] gitaly[899]: time="2026-11-12T09:23:28.406Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3EZB9SN2NF6DFEQ212P2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:23:58.005" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:28.405" grpc.time_ms=0.42 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1901gitlab # [ 253.431472] gitaly[899]: time="2026-11-12T09:23:28.465Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3EZB9SN2NF6DFEQ212P2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:38.117" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:28.458" grpc.time_ms=7.004 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1902gitlab # [ 253.471608] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:28 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/85399d301bf194486bc26d409ef791d202275c25 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 1411903gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7'1904gitlab # [ 253.528864] gitaly[899]: time="2026-11-12T09:23:28.562Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3F43FNPFZT8JRFBGRXD7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:23:58.461" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:28.561" grpc.time_ms=0.698 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1905gitlab # [ 253.588465] nixbot[2660]: {"ts": "2026-11-12T09:23:28.622Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47216 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 HTTP/1.1\" 200"}1906gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', in 0.04 seconds)1907gitlab # [ 253.636508] gitaly[899]: time="2026-11-12T09:23:28.670Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M77H3F43FNPFZT8JRFBGRXD7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:38.253" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:28.663" grpc.time_ms=6.474 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1908gitlab # [ 253.671455] gitaly[899]: time="2026-11-12T09:23:28.705Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-12T09:23:58.695" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:28.695" grpc.time_ms=9.517 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1909gitlab # [ 253.712698] gitaly[899]: time="2026-11-12T09:23:28.746Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3980EJG8C63ZS7A961EP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-12T09:23:58.735" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:28.735" grpc.time_ms=11.158 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1910gitlab # [ 253.713731] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:28 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/85399d301bf194486bc26d409ef791d202275c25 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2311911gitlab # [ 253.725636] sidekiq-cluster[2534]: {"severity":"WARN","time":"2026-11-12T09:23:28.759Z","class":"CreatePipelineWorker","correlation_id":"01M77H3980EJG8C63ZS7A961EP","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"85399d301bf194486bc26d409ef791d202275c25","after":"e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7","ref":"refs/heads/master"},"retry":0}1912gitlab # [ 253.729193] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:28.762Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"482bebb908737aadbe460ce3","created_at":"2026-11-12T09:23:26.152Z","trace_propagation_headers":{"sentry-trace":"009f533470024edbb821b9e5537f7512-c642a8b4a96a4a69","baggage":"sentry-trace_id=009f533470024edbb821b9e5537f7512,sentry-release=Unknown"},"correlation_id":"01M77H3980EJG8C63ZS7A961EP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b89c2c349a4d3c6d2bda7ef12bbc8d520ded6357aa04072972e66dea320947ed","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:26.156Z","job_size_bytes":467,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-482bebb908737aadbe460ce3: done: 0.854461 sec","job_status":"done","queue_duration_s":1.752433,"scheduling_latency_s":1.752448,"gitaly_calls":7,"gitaly_duration_s":0.381098,"redis_calls":33,"redis_duration_s":0.003607,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000454,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000664,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001871,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000618,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.094,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.092,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.373525,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.854461,"completed_at":"2026-11-12T09:23:28.762Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.093971,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1913gitlab # [ 253.737330] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:28.764Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"459e8b1f-975a-41d4-80af-c49ea2148eed\"}"],"class":"WebHookWorker","jid":"74c51319fabc339bb2813e6a","created_at":"2026-11-12T09:23:26.175Z","trace_propagation_headers":{"sentry-trace":"009f533470024edbb821b9e5537f7512-c642a8b4a96a4a69","baggage":"sentry-trace_id=009f533470024edbb821b9e5537f7512,sentry-release=Unknown"},"correlation_id":"01M77H3980EJG8C63ZS7A961EP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:54db39dea440e0b202cb815e7b60babcf30e2c526dee4be5d8262fccb75b3ca7","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:26.178Z","job_size_bytes":1749,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-74c51319fabc339bb2813e6a: start","job_status":"start","queue_duration_s":2.586125,"scheduling_latency_s":2.586143}1914gitlab # [ 253.754004] nixbot[2660]: {"ts": "2026-11-12T09:23:28.781Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "8e664b57-07d2-4227-9dce-561148bc65c8", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7", "pr": null}1915gitlab # [ 253.757133] nixbot[2660]: {"ts": "2026-11-12T09:23:28.783Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47224 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}1916gitlab # [ 253.761513] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:28.794Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"459e8b1f-975a-41d4-80af-c49ea2148eed\"}"],"class":"WebHookWorker","jid":"74c51319fabc339bb2813e6a","created_at":"2026-11-12T09:23:26.175Z","trace_propagation_headers":{"sentry-trace":"009f533470024edbb821b9e5537f7512-c642a8b4a96a4a69","baggage":"sentry-trace_id=009f533470024edbb821b9e5537f7512,sentry-release=Unknown"},"correlation_id":"01M77H3980EJG8C63ZS7A961EP","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:54db39dea440e0b202cb815e7b60babcf30e2c526dee4be5d8262fccb75b3ca7","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:26.178Z","job_size_bytes":1749,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-74c51319fabc339bb2813e6a: done: 0.030049 sec","job_status":"done","queue_duration_s":2.586125,"scheduling_latency_s":2.586143,"redis_calls":14,"redis_duration_s":0.003778,"redis_read_bytes":25,"redis_write_bytes":4683,"redis_queues_calls":2,"redis_queues_duration_s":0.001088,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3727,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002013,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000677,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.006477080999985674,"cpu_s":0.015623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.030049,"completed_at":"2026-11-12T09:23:28.794Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001097,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1917gitlab # [ 253.826206] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:28.795Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"4740e5383637026293398d4b","created_at":"2026-11-12T09:23:26.293Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-7e9e5aedf34d4bfe","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:26.296Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-4740e5383637026293398d4b: start","job_status":"start","queue_duration_s":2.499087,"scheduling_latency_s":2.499104}1918gitlab # [ 253.827039] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:28.823Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"4740e5383637026293398d4b","created_at":"2026-11-12T09:23:26.293Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-7e9e5aedf34d4bfe","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:26.296Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-4740e5383637026293398d4b: done: 0.028487 sec","job_status":"done","queue_duration_s":2.499087,"scheduling_latency_s":2.499104,"redis_calls":19,"redis_duration_s":0.002539,"redis_read_bytes":21,"redis_write_bytes":4290,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00059,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000344,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2261,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001168,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.004585219000000001,"duration_s":0.028487,"completed_at":"2026-11-12T09:23:28.823Z","load_balancing_strategy":"primary","db_duration_s":0.00438,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1919gitlab # [ 253.884878] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:28.824Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","b42ad2aaf17d378fd1746b9c"],"class":"WebHooks::LogExecutionWorker","jid":"073a4912a8a85898f45127ee","created_at":"2026-11-12T09:23:27.192Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-44e1e3e8bf584d6f","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:75f83f72e07ff655c8b50ba69814dca27786112054725c50ebd4f5d38b62f925","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:27.200Z","job_size_bytes":2593,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-073a4912a8a85898f45127ee: start","job_status":"start","queue_duration_s":1.623803,"scheduling_latency_s":1.623816}1920gitlab # [ 253.885728] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:28.853Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","b42ad2aaf17d378fd1746b9c"],"class":"WebHooks::LogExecutionWorker","jid":"073a4912a8a85898f45127ee","created_at":"2026-11-12T09:23:27.192Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-44e1e3e8bf584d6f","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:75f83f72e07ff655c8b50ba69814dca27786112054725c50ebd4f5d38b62f925","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:27.200Z","job_size_bytes":2593,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-073a4912a8a85898f45127ee: done: 0.028258 sec","job_status":"done","queue_duration_s":1.623803,"scheduling_latency_s":1.623816,"redis_calls":10,"redis_duration_s":0.00133,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000267,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000813,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00025,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019203,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000226286,"duration_s":0.028258,"completed_at":"2026-11-12T09:23:28.852Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005191,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1921gitlab # [ 253.944600] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:28.854Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3af0d3c73e59da6d07d67e22","created_at":"2026-11-12T09:23:28.491Z","trace_propagation_headers":{"sentry-trace":"f916403c41e241d9afb3a69d5e94b1e0-379e1060853e47cd","baggage":"sentry-trace_id=f916403c41e241d9afb3a69d5e94b1e0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H3EZB9SN2NF6DFEQ212P2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:28.494Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3af0d3c73e59da6d07d67e22: start","job_status":"start","queue_duration_s":0.3594,"scheduling_latency_s":0.359413}1922gitlab # [ 253.945792] gitaly[899]: time="2026-11-12T09:23:28.904Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3EZB9SN2NF6DFEQ212P2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-12T09:23:58.002" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:28.903" grpc.time_ms=0.837 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1923gitlab # [ 253.972149] gitaly[899]: time="2026-11-12T09:23:29.006Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3EZB9SN2NF6DFEQ212P2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:23:59.005" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:29.005" grpc.time_ms=0.515 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1924gitlab # [ 253.980194] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.014Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3af0d3c73e59da6d07d67e22","created_at":"2026-11-12T09:23:28.491Z","trace_propagation_headers":{"sentry-trace":"f916403c41e241d9afb3a69d5e94b1e0-379e1060853e47cd","baggage":"sentry-trace_id=f916403c41e241d9afb3a69d5e94b1e0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H3EZB9SN2NF6DFEQ212P2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:28.494Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2534,"job_status":"rescheduled","message":"PipelineProcessWorker JID-3af0d3c73e59da6d07d67e22: rescheduled"}1925gitlab # [ 253.985402] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.019Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"3af0d3c73e59da6d07d67e22","created_at":"2026-11-12T09:23:28.491Z","trace_propagation_headers":{"sentry-trace":"f916403c41e241d9afb3a69d5e94b1e0-379e1060853e47cd","baggage":"sentry-trace_id=f916403c41e241d9afb3a69d5e94b1e0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H3EZB9SN2NF6DFEQ212P2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:28.494Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-3af0d3c73e59da6d07d67e22: done: 0.165376 sec","job_status":"done","queue_duration_s":0.3594,"scheduling_latency_s":0.359413,"gitaly_calls":2,"gitaly_duration_s":0.006028,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.0044989999999999995,"redis_read_bytes":59,"redis_write_bytes":6216,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000603,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000564,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_calls":4,"redis_queues_duration_s":0.000408,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2219,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001223,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":979,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000153,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000144,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000801,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":29,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.093,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":14,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.085,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064433,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.140260843,"duration_s":0.165376,"completed_at":"2026-11-12T09:23:29.019Z","load_balancing_strategy":"primary","db_duration_s":0.095657,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1926gitlab # [ 254.008778] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.020Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","74c51319fabc339bb2813e6a"],"class":"WebHooks::LogExecutionWorker","jid":"5c966fd455242237c346bcb6","created_at":"2026-11-12T09:23:28.785Z","trace_propagation_headers":{"sentry-trace":"009f533470024edbb821b9e5537f7512-8351905c56464abf","baggage":"sentry-trace_id=009f533470024edbb821b9e5537f7512,sentry-release=Unknown"},"correlation_id":"01M77H3980EJG8C63ZS7A961EP","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:97997e75d922ad4491c4b3bceab1d4e83f2ab4349f9cc57118f7a3187f639f1c","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:28.787Z","job_size_bytes":2592,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-5c966fd455242237c346bcb6: start","job_status":"start","queue_duration_s":0.232784,"scheduling_latency_s":0.232799}1927gitlab # [ 254.024197] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.038Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","74c51319fabc339bb2813e6a"],"class":"WebHooks::LogExecutionWorker","jid":"5c966fd455242237c346bcb6","created_at":"2026-11-12T09:23:28.785Z","trace_propagation_headers":{"sentry-trace":"009f533470024edbb821b9e5537f7512-8351905c56464abf","baggage":"sentry-trace_id=009f533470024edbb821b9e5537f7512,sentry-release=Unknown"},"correlation_id":"01M77H3980EJG8C63ZS7A961EP","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:97997e75d922ad4491c4b3bceab1d4e83f2ab4349f9cc57118f7a3187f639f1c","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:28.787Z","job_size_bytes":2592,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-5c966fd455242237c346bcb6: done: 0.01774 sec","job_status":"done","queue_duration_s":0.232784,"scheduling_latency_s":0.232799,"redis_calls":7,"redis_duration_s":0.000929,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000666,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01774,"completed_at":"2026-11-12T09:23:29.038Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00335,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1928gitlab # [ 254.081934] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.039Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"b9e75804bd9767f453dd43a1","created_at":"2026-11-12T09:23:28.815Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-89293aeed850425c","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:28.818Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-b9e75804bd9767f453dd43a1: start","job_status":"start","queue_duration_s":0.220467,"scheduling_latency_s":0.220482}1929gitlab # [ 254.082628] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.093Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"b9e75804bd9767f453dd43a1","created_at":"2026-11-12T09:23:28.815Z","trace_propagation_headers":{"sentry-trace":"7c54cb3f8faa45758212aae194a2bc7b-89293aeed850425c","baggage":"sentry-trace_id=7c54cb3f8faa45758212aae194a2bc7b,sentry-release=Unknown"},"correlation_id":"01M77H1V0BNZ4JGW1XVJEH6DKC","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:28.818Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-b9e75804bd9767f453dd43a1: done: 0.054373 sec","job_status":"done","queue_duration_s":0.220467,"scheduling_latency_s":0.220482,"redis_calls":11,"redis_duration_s":0.00152,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000285,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208685,"duration_s":0.054373,"completed_at":"2026-11-12T09:23:29.093Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012614,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1930gitlab # [ 254.151179] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.094Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"edca565294ceed5b28516797","created_at":"2026-11-12T09:23:28.892Z","trace_propagation_headers":{"sentry-trace":"f916403c41e241d9afb3a69d5e94b1e0-9e6ef5b6b24046fb","baggage":"sentry-trace_id=f916403c41e241d9afb3a69d5e94b1e0,sentry-release=Unknown"},"correlation_id":"01M77H3EZB9SN2NF6DFEQ212P2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:28.893Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-edca565294ceed5b28516797: start","job_status":"start","queue_duration_s":0.200814,"scheduling_latency_s":0.200827}1931gitlab # [ 254.151818] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.115Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"edca565294ceed5b28516797","created_at":"2026-11-12T09:23:28.892Z","trace_propagation_headers":{"sentry-trace":"f916403c41e241d9afb3a69d5e94b1e0-9e6ef5b6b24046fb","baggage":"sentry-trace_id=f916403c41e241d9afb3a69d5e94b1e0,sentry-release=Unknown"},"correlation_id":"01M77H3EZB9SN2NF6DFEQ212P2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:28.893Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-edca565294ceed5b28516797: done: 0.020526 sec","job_status":"done","queue_duration_s":0.200814,"scheduling_latency_s":0.200827,"redis_calls":7,"redis_duration_s":0.0009419999999999999,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020526,"completed_at":"2026-11-12T09:23:29.115Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003041,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1932gitlab # [ 254.195453] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.116Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8ce5462a2c1a750524f6257c","created_at":"2026-11-12T09:23:29.014Z","trace_propagation_headers":{"sentry-trace":"f916403c41e241d9afb3a69d5e94b1e0-9e6ef5b6b24046fb","baggage":"sentry-trace_id=f916403c41e241d9afb3a69d5e94b1e0,sentry-release=Unknown"},"correlation_id":"01M77H3EZB9SN2NF6DFEQ212P2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.015Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8ce5462a2c1a750524f6257c: start","job_status":"start","queue_duration_s":0.10024,"scheduling_latency_s":0.100256}1933gitlab # [ 254.196143] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.131Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8ce5462a2c1a750524f6257c","created_at":"2026-11-12T09:23:29.014Z","trace_propagation_headers":{"sentry-trace":"f916403c41e241d9afb3a69d5e94b1e0-9e6ef5b6b24046fb","baggage":"sentry-trace_id=f916403c41e241d9afb3a69d5e94b1e0,sentry-release=Unknown"},"correlation_id":"01M77H3EZB9SN2NF6DFEQ212P2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.015Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8ce5462a2c1a750524f6257c: done: 0.015179 sec","job_status":"done","queue_duration_s":0.10024,"scheduling_latency_s":0.100256,"redis_calls":8,"redis_duration_s":0.001057,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000817,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00024,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015179,"completed_at":"2026-11-12T09:23:29.131Z","load_balancing_strategy":"primary","db_duration_s":0.002358,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1934gitlab # [ 254.303790] gitaly[899]: time="2026-11-12T09:23:29.337Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3FWF7Y0VJ9PSCNKAHYA8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.331" grpc.time_ms=6.584 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1935gitlab # [ 254.316988] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:29 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 551936gitlab # [ 254.320398] gitaly[899]: time="2026-11-12T09:23:29.354Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3FWF7Y0VJ9PSCP0NMN22 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:23:59.253" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:29.353" grpc.time_ms=0.725 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1937gitlab # [ 254.375338] gitaly[899]: time="2026-11-12T09:23:29.409Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M77H3FWF7Y0VJ9PSCP0NMN22 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:39.229" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:29.400" grpc.time_ms=8.791 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1938gitlab # [ 254.468155] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.501Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"d18a6479ee5b75b8380a2caf","created_at":"2026-11-12T09:23:29.489Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-fe287bbc9dc84f27","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.497Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d18a6479ee5b75b8380a2caf: start","job_status":"start","queue_duration_s":0.003525,"scheduling_latency_s":0.003634}1939gitlab # [ 254.501949] gitaly[899]: time="2026-11-12T09:23:29.535Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M77H3FY9R8JRS1B9X1GTX2JC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.495" pid=899 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1940gitlab # [ 254.503760] gitaly[899]: time="2026-11-12T09:23:29.537Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=36 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3FY9R8JRS1B9X1GTX2JC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.495" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1941gitlab # [ 254.504792] gitaly[899]: time="2026-11-12T09:23:29.537Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=36 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3FY9R8JRS1B9X1GTX2JC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.495" grpc.time_ms=41.558 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1942gitlab # [ 254.536184] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:29 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/85399d301bf194486bc26d409ef791d202275c25 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2751943gitlab # [ 254.538270] gitlab-workhorse[825]: time="2026-11-12T09:23:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671944gitlab # [ 254.539877] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:29 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 2181945gitlab # [ 254.580225] gitaly[899]: time="2026-11-12T09:23:29.614Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H3FWF7Y0VJ9PSCP0NMN22 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:23:29.613" grpc.time_ms=0.33 pid=899 span.kind=server system=grpc1946gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7'1947gitlab # [ 254.606127] gitaly[899]: time="2026-11-12T09:23:29.639Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H3FWF7Y0VJ9PSCP0NMN22 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:23:29.638" grpc.time_ms=0.548 pid=899 span.kind=server system=grpc1948gitlab # [ 254.626343] gitaly[899]: time="2026-11-12T09:23:29.655Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=8 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M77H3G63V8H94VMFPYP3HPFY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.640" grpc.time_ms=15.315 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1949gitlab # [ 254.627113] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:29 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 571950gitlab # [ 254.640306] gitaly[899]: time="2026-11-12T09:23:29.674Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H3FWF7Y0VJ9PSCP0NMN22 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:23:29.672" grpc.time_ms=0.883 pid=899 span.kind=server system=grpc1951gitlab # [ 254.656792] nixbot[2660]: {"ts": "2026-11-12T09:23:29.690Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47228 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 HTTP/1.1\" 200"}1952gitlab # [ 254.661074] gitaly[899]: time="2026-11-12T09:23:29.694Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M77H3FWF7Y0VJ9PSCP0NMN22 diskcache=4d3c554d-83e9-4828-8f21-2dc3f3af386c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-12T09:23:59.590" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:29.590" pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1953gitlab # [ 254.661634] gitaly[899]: time="2026-11-12T09:23:29.695Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=78 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7335 command.oublock=0 command.real_time_ms=93 command.spawn_token_fork_ms=2 command.system_time_ms=62 command.user_time_ms=16 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3FWF7Y0VJ9PSCP0NMN22 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-12T09:23:59.590" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:29.590" grpc.time_ms=104.235 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1954gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', in 0.09 seconds)1955gitlab # [ 254.688220] gitaly[899]: time="2026-11-12T09:23:29.722Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M77H3G7YETKT64R8VHDG0F80 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.714" pid=899 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1956gitlab # [ 254.697715] gitaly[899]: time="2026-11-12T09:23:29.722Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3G7YETKT64R8VHDG0F80 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.714" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1957gitlab # [ 254.710436] gitlab-workhorse[825]: time="2026-11-12T09:23:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671958gitlab # [ 254.712269] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:29 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 601959gitlab # [ 254.714262] gitaly[899]: time="2026-11-12T09:23:29.722Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3G7YETKT64R8VHDG0F80 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.714" grpc.time_ms=8.333 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1960gitlab # [ 254.727192] gitaly[899]: time="2026-11-12T09:23:29.741Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3FWF7Y0VJ9PSCP0NMN22 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-12T09:23:59.740" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:29.740" grpc.time_ms=0.512 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1961gitlab # [ 254.738591] gitaly[899]: time="2026-11-12T09:23:29.768Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3FWF7Y0VJ9PSCP0NMN22 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:23:59.767" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:29.767" grpc.time_ms=0.511 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1962gitlab # [ 254.750268] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.779Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"d18a6479ee5b75b8380a2caf","created_at":"2026-11-12T09:23:29.489Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-fe287bbc9dc84f27","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.497Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d18a6479ee5b75b8380a2caf: done: 0.278618 sec","job_status":"done","queue_duration_s":0.003525,"scheduling_latency_s":0.003634,"gitaly_calls":3,"gitaly_duration_s":0.110667,"redis_calls":71,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.014055,"redis_read_bytes":151,"redis_write_bytes":19877,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00095,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00161,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":23,"redis_queues_duration_s":0.002491,"redis_queues_read_bytes":25,"redis_queues_write_bytes":13379,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.004626,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3094,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000254,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000413,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002761,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.112059,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.232412778,"duration_s":0.278618,"completed_at":"2026-11-12T09:23:29.779Z","load_balancing_strategy":"primary","db_duration_s":0.042527,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1963gitlab # [ 254.795925] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.780Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9442b1413aec4d923cc2b805","created_at":"2026-11-12T09:23:29.580Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.582Z","job_size_bytes":23,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9442b1413aec4d923cc2b805: start","job_status":"start","queue_duration_s":0.19827,"scheduling_latency_s":0.198291}1964gitlab # [ 254.810746] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.807Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9442b1413aec4d923cc2b805","created_at":"2026-11-12T09:23:29.580Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.582Z","job_size_bytes":23,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9442b1413aec4d923cc2b805: done: 0.026868 sec","job_status":"done","queue_duration_s":0.19827,"scheduling_latency_s":0.198291,"redis_calls":7,"redis_duration_s":0.0010279999999999998,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000748,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00028,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026868,"completed_at":"2026-11-12T09:23:29.807Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004697,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1965gitlab # [ 254.848988] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.808Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"30c7c894c542e4992c3682b8","created_at":"2026-11-12T09:23:29.699Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.702Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-30c7c894c542e4992c3682b8: start","job_status":"start","queue_duration_s":0.10597,"scheduling_latency_s":0.105986}1966gitlab # [ 254.865234] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:29 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 441967gitlab # [ 254.867296] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:29 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 491968gitlab # [ 254.869315] gitlab-workhorse[825]: time="2026-11-12T09:23:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671969gitlab # [ 254.871062] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:29 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 471970gitlab # [ 254.872908] gitlab-workhorse[825]: time="2026-11-12T09:23:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2211971gitlab # [ 254.874682] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:29 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 221 "" "git/2.55.0" 551972gitlab # [ 254.876548] gitaly[899]: time="2026-11-12T09:23:29.794Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3GAQCVDC02D8K8N5W0W3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.789" grpc.time_ms=4.941 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1973gitlab # [ 254.888486] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.825Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"30c7c894c542e4992c3682b8","created_at":"2026-11-12T09:23:29.699Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.702Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-30c7c894c542e4992c3682b8: done: 0.016922 sec","job_status":"done","queue_duration_s":0.10597,"scheduling_latency_s":0.105986,"redis_calls":7,"redis_duration_s":0.001029,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000777,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000252,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016922,"completed_at":"2026-11-12T09:23:29.825Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1974gitlab # [ 254.926968] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.826Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"073fc6ef4f884ebfde2896b7","created_at":"2026-11-12T09:23:29.708Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.711Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-073fc6ef4f884ebfde2896b7: start","job_status":"start","queue_duration_s":0.114757,"scheduling_latency_s":0.114771}1975gitlab # [ 254.927698] gitaly[899]: time="2026-11-12T09:23:29.808Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3GAYW794BC587B3YJQX9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.801" grpc.time_ms=6.642 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1976gitlab # [ 254.928478] gitaly[899]: time="2026-11-12T09:23:29.845Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GC7FPCJNRTA613R66KC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.839" pid=899 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1977gitlab # [ 254.929061] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:29 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 511978gitlab # [ 254.929519] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.844Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"073fc6ef4f884ebfde2896b7","created_at":"2026-11-12T09:23:29.708Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.711Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-073fc6ef4f884ebfde2896b7: done: 0.017736 sec","job_status":"done","queue_duration_s":0.114757,"scheduling_latency_s":0.114771,"redis_calls":7,"redis_duration_s":0.002042,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000846,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001196,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017736,"completed_at":"2026-11-12T09:23:29.844Z","load_balancing_strategy":"primary","db_duration_s":0.002583,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1979gitlab # [ 254.969181] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.845Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"812db406a70006f863db140f","created_at":"2026-11-12T09:23:29.717Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.718Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-812db406a70006f863db140f: start","job_status":"start","queue_duration_s":0.126765,"scheduling_latency_s":0.12678}1980gitlab # [ 254.974402] gitaly[899]: time="2026-11-12T09:23:29.845Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GC7FPCJNRTA613R66KC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.839" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1981gitlab # [ 255.015093] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.865Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"812db406a70006f863db140f","created_at":"2026-11-12T09:23:29.717Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.718Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-812db406a70006f863db140f: done: 0.020285 sec","job_status":"done","queue_duration_s":0.126765,"scheduling_latency_s":0.12678,"redis_calls":10,"redis_duration_s":0.001315,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000708,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000361,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020285,"completed_at":"2026-11-12T09:23:29.865Z","load_balancing_strategy":"primary","db_duration_s":0.003185,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1982gitlab # [ 255.023862] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.866Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"fea0d5c5e5a5e5f07448e1c6","created_at":"2026-11-12T09:23:29.718Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.719Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-fea0d5c5e5a5e5f07448e1c6: start","job_status":"start","queue_duration_s":0.14668,"scheduling_latency_s":0.146696}1983gitlab # [ 255.066869] gitaly[899]: time="2026-11-12T09:23:29.846Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GC7FPCJNRTA613R66KC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.839" grpc.time_ms=6.658 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1984gitlab # [ 255.067644] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.883Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"fea0d5c5e5a5e5f07448e1c6","created_at":"2026-11-12T09:23:29.718Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.719Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-fea0d5c5e5a5e5f07448e1c6: done: 0.016623 sec","job_status":"done","queue_duration_s":0.14668,"scheduling_latency_s":0.146696,"redis_calls":7,"redis_duration_s":0.001738,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001051,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000687,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010532,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016623,"completed_at":"2026-11-12T09:23:29.883Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002542,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1985gitlab # [ 255.110137] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.885Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ba27bb863f964a1dd3e8da6f","created_at":"2026-11-12T09:23:29.720Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.721Z","job_size_bytes":19,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ba27bb863f964a1dd3e8da6f: start","job_status":"start","queue_duration_s":0.164073,"scheduling_latency_s":0.164089}1986gitlab # [ 255.110824] gitaly[899]: time="2026-11-12T09:23:29.865Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GCJQH9W14R9JBPW8RS7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.846" pid=899 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=221 span.kind=server system=grpc1987gitlab # [ 255.111462] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.902Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ba27bb863f964a1dd3e8da6f","created_at":"2026-11-12T09:23:29.720Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.721Z","job_size_bytes":19,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ba27bb863f964a1dd3e8da6f: done: 0.01665 sec","job_status":"done","queue_duration_s":0.164073,"scheduling_latency_s":0.164089,"redis_calls":8,"redis_duration_s":0.0024519999999999998,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002084,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000368,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009735,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01665,"completed_at":"2026-11-12T09:23:29.902Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002122,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1988gitlab # [ 255.183828] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.903Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"4ee6f9420df922953594247f","created_at":"2026-11-12T09:23:29.722Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.723Z","job_size_bytes":66,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-4ee6f9420df922953594247f: start","job_status":"start","queue_duration_s":0.180125,"scheduling_latency_s":0.18014}1989gitlab # [ 255.184688] gitaly[899]: time="2026-11-12T09:23:29.865Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1172 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GCJQH9W14R9JBPW8RS7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.846" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1990gitlab # [ 255.185463] gitlab-workhorse[825]: time="2026-11-12T09:23:30Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2211991gitlab # [ 255.185863] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:30 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 221 "" "git/2.55.0" 2381992gitlab # [ 255.186525] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.917Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"4ee6f9420df922953594247f","created_at":"2026-11-12T09:23:29.722Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.723Z","job_size_bytes":66,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-4ee6f9420df922953594247f: done: 0.013926 sec","job_status":"done","queue_duration_s":0.180125,"scheduling_latency_s":0.18014,"redis_calls":7,"redis_duration_s":0.001281,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000995,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000286,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013926,"completed_at":"2026-11-12T09:23:29.917Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00154,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1993gitlab # [ 255.238805] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.918Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"56b57f778a001bc155084ccf","created_at":"2026-11-12T09:23:29.723Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.724Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-56b57f778a001bc155084ccf: start","job_status":"start","queue_duration_s":0.193428,"scheduling_latency_s":0.193442}1994gitlab # [ 255.239696] gitaly[899]: time="2026-11-12T09:23:29.865Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1172 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GCJQH9W14R9JBPW8RS7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.846" grpc.time_ms=19.161 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1995gitlab # [ 255.249027] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.942Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"56b57f778a001bc155084ccf","created_at":"2026-11-12T09:23:29.723Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.724Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-56b57f778a001bc155084ccf: done: 0.024542 sec","job_status":"done","queue_duration_s":0.193428,"scheduling_latency_s":0.193442,"redis_calls":7,"redis_duration_s":0.001156,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024542,"completed_at":"2026-11-12T09:23:29.942Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003622,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1996gitlab # [ 255.290744] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.943Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"48ae2e97a0f09ad1285c6b46","created_at":"2026-11-12T09:23:29.727Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.728Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-48ae2e97a0f09ad1285c6b46: start","job_status":"start","queue_duration_s":0.215468,"scheduling_latency_s":0.215483}1997gitlab # [ 255.303259] gitaly[899]: time="2026-11-12T09:23:29.934Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3GEV1GQ51DJMWD4B0YWB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=202 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:29.921" grpc.time_ms=12.635 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc1998gitlab # [ 255.315384] gitaly[899]: time="2026-11-12T09:23:30.174Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GGGHHR11E8W9GST7H06 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:30.163" pid=899 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=221 span.kind=server system=grpc1999gitlab # [ 255.360387] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:29.960Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"48ae2e97a0f09ad1285c6b46","created_at":"2026-11-12T09:23:29.727Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.728Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-48ae2e97a0f09ad1285c6b46: done: 0.017283 sec","job_status":"done","queue_duration_s":0.215468,"scheduling_latency_s":0.215483,"redis_calls":7,"redis_duration_s":0.001511,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001243,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017283,"completed_at":"2026-11-12T09:23:29.960Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004004,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2000gitlab # [ 255.402451] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:30.150Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"92bc4ab3be823b986be40f74","created_at":"2026-11-12T09:23:29.728Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.729Z","job_size_bytes":6,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-92bc4ab3be823b986be40f74: start","job_status":"start","queue_duration_s":0.420953,"scheduling_latency_s":0.420992}2001gitlab # [ 255.402975] gitaly[899]: time="2026-11-12T09:23:30.174Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GGGHHR11E8W9GST7H06 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:30.163" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc2002gitlab # [ 255.404027] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 2282003gitlab # [ 255.404749] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:30.163Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"92bc4ab3be823b986be40f74","created_at":"2026-11-12T09:23:29.728Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.729Z","job_size_bytes":6,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-92bc4ab3be823b986be40f74: done: 0.013327 sec","job_status":"done","queue_duration_s":0.420953,"scheduling_latency_s":0.420992,"redis_calls":7,"redis_duration_s":0.001532,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001273,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013327,"completed_at":"2026-11-12T09:23:30.163Z","load_balancing_strategy":"primary","db_duration_s":0.000414,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2004gitlab # [ 255.445156] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:30.164Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"cbb3d09ae8a896737f630465","created_at":"2026-11-12T09:23:29.729Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.730Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-cbb3d09ae8a896737f630465: start","job_status":"start","queue_duration_s":0.433995,"scheduling_latency_s":0.434008}2005gitlab # [ 255.445910] gitaly[899]: time="2026-11-12T09:23:30.174Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1169 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GGGHHR11E8W9GST7H06 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:30.163" grpc.time_ms=11.686 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc2006gitlab # [ 255.459257] nixbot[2660]: {"ts": "2026-11-12T09:23:30.445Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-e26299576e5f-6"}2007gitlab # [ 255.460786] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:30.183Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"cbb3d09ae8a896737f630465","created_at":"2026-11-12T09:23:29.729Z","trace_propagation_headers":{"sentry-trace":"41af5e27ba4c41de886e62692288010e-5909beab9aaa4fc5","baggage":"sentry-trace_id=41af5e27ba4c41de886e62692288010e,sentry-release=Unknown"},"correlation_id":"01M77H3FWF7Y0VJ9PSCP0NMN22","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:29.730Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-cbb3d09ae8a896737f630465: done: 0.019504 sec","job_status":"done","queue_duration_s":0.433995,"scheduling_latency_s":0.434008,"redis_calls":7,"redis_duration_s":0.001373,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000509,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019504,"completed_at":"2026-11-12T09:23:30.183Z","load_balancing_strategy":"primary","db_duration_s":0.003937,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2008gitlab # [ 255.517095] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:30.373Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"2b342177d7023044c54ff733","created_at":"2026-11-12T09:23:30.368Z","trace_propagation_headers":{"sentry-trace":"99b8cc46c48947119e3da7bcec0bff79-8dcc41d2b8594a77","baggage":"sentry-trace_id=99b8cc46c48947119e3da7bcec0bff79,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H3GRVZ8VAJ57BKTCPH8ZB","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:30.372Z","job_size_bytes":65,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-2b342177d7023044c54ff733: start","job_status":"start","queue_duration_s":0.001269,"scheduling_latency_s":0.001305}2009gitlab # [ 255.518565] gitaly[899]: time="2026-11-12T09:23:30.248Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GRVZ8VAJ57BKTCPH8ZB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:24:00.047" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:30.247" grpc.time_ms=0.576 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2010gitlab # [ 255.524719] nixbot[2660]: {"ts": "2026-11-12T09:23:30.538Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-e26299576e5f-6", "argv0": "bwrap"}2011gitlab # [ 255.526682] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:30.401Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"2b342177d7023044c54ff733","created_at":"2026-11-12T09:23:30.368Z","trace_propagation_headers":{"sentry-trace":"99b8cc46c48947119e3da7bcec0bff79-8dcc41d2b8594a77","baggage":"sentry-trace_id=99b8cc46c48947119e3da7bcec0bff79,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H3GRVZ8VAJ57BKTCPH8ZB","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:30.372Z","job_size_bytes":65,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-2b342177d7023044c54ff733: done: 0.027547 sec","job_status":"done","queue_duration_s":0.001269,"scheduling_latency_s":0.001305,"redis_calls":7,"redis_duration_s":0.001144,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000862,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000282,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027547,"completed_at":"2026-11-12T09:23:30.401Z","load_balancing_strategy":"primary","db_duration_s":0.004329,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2012gitlab # [ 255.593721] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:30.402Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"3990c4d6cdbe6b5c8cdb5fd0","created_at":"2026-11-12T09:23:30.372Z","trace_propagation_headers":{"sentry-trace":"99b8cc46c48947119e3da7bcec0bff79-8dcc41d2b8594a77","baggage":"sentry-trace_id=99b8cc46c48947119e3da7bcec0bff79,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H3GRVZ8VAJ57BKTCPH8ZB","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:30.374Z","job_size_bytes":65,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-3990c4d6cdbe6b5c8cdb5fd0: start","job_status":"start","queue_duration_s":0.027587,"scheduling_latency_s":0.027636}2013gitlab # [ 255.594522] gitaly[899]: time="2026-11-12T09:23:30.264Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3GRVZ8VAJ57BKTCPH8ZB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-12T09:24:00.057" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:30.258" grpc.time_ms=6.492 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2014gitlab # [ 255.595115] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:30.438Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"3990c4d6cdbe6b5c8cdb5fd0","created_at":"2026-11-12T09:23:30.372Z","trace_propagation_headers":{"sentry-trace":"99b8cc46c48947119e3da7bcec0bff79-8dcc41d2b8594a77","baggage":"sentry-trace_id=99b8cc46c48947119e3da7bcec0bff79,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H3GRVZ8VAJ57BKTCPH8ZB","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:30.374Z","job_size_bytes":65,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-3990c4d6cdbe6b5c8cdb5fd0: done: 0.036146 sec","job_status":"done","queue_duration_s":0.027587,"scheduling_latency_s":0.027636,"redis_calls":10,"redis_duration_s":0.0017789999999999998,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036146,"completed_at":"2026-11-12T09:23:30.438Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009453,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2015gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7'2016gitlab # [ 255.668689] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:30.439Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"5dcc04bbe511a188d45a3887","created_at":"2026-11-12T09:23:30.418Z","trace_propagation_headers":{"sentry-trace":"99b8cc46c48947119e3da7bcec0bff79-8dcc41d2b8594a77","baggage":"sentry-trace_id=99b8cc46c48947119e3da7bcec0bff79,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H3GRVZ8VAJ57BKTCPH8ZB","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:30.421Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5dcc04bbe511a188d45a3887: start","job_status":"start","queue_duration_s":0.018306,"scheduling_latency_s":0.018323}2017gitlab # [ 255.669528] gitaly[899]: time="2026-11-12T09:23:30.274Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M77H3GRVZ8VAJ57BKTCPH8ZB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:40.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:30.269" grpc.time_ms=5.38 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2018gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', in 0.05 seconds)2019(finished: subtest: push delivers the webhook, in 52.21 seconds)2020subtest: eval and build post commit statuses2021gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses2022gitlab # [ 255.670191] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:30.528Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"5dcc04bbe511a188d45a3887","created_at":"2026-11-12T09:23:30.418Z","trace_propagation_headers":{"sentry-trace":"99b8cc46c48947119e3da7bcec0bff79-8dcc41d2b8594a77","baggage":"sentry-trace_id=99b8cc46c48947119e3da7bcec0bff79,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M77H3GRVZ8VAJ57BKTCPH8ZB","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:30.421Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5dcc04bbe511a188d45a3887: done: 0.089198 sec","job_status":"done","queue_duration_s":0.018306,"scheduling_latency_s":0.018323,"gitaly_calls":2,"gitaly_duration_s":0.005361,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.004328,"redis_read_bytes":79,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000587,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000726,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.00021,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001061,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000144,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000282,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000731,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061875,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.066352009,"duration_s":0.089198,"completed_at":"2026-11-12T09:23:30.528Z","load_balancing_strategy":"primary","db_duration_s":0.021292,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2023gitlab # [ 255.726109] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:30.529Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"30d72af0468ae640c27c6074","created_at":"2026-11-12T09:23:30.479Z","trace_propagation_headers":{"sentry-trace":"99b8cc46c48947119e3da7bcec0bff79-89de2d60915a48bf","baggage":"sentry-trace_id=99b8cc46c48947119e3da7bcec0bff79,sentry-release=Unknown"},"correlation_id":"01M77H3GRVZ8VAJ57BKTCPH8ZB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:30.481Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-30d72af0468ae640c27c6074: start","job_status":"start","queue_duration_s":0.048785,"scheduling_latency_s":0.048799}2024gitlab # [ 255.726818] gitaly[899]: time="2026-11-12T09:23:30.296Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GRVZ8VAJ57BKTCPH8ZB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-12T09:23:40.035" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:23:30.295" grpc.time_ms=0.195 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2025gitlab # [ 255.727552] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:30.557Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"30d72af0468ae640c27c6074","created_at":"2026-11-12T09:23:30.479Z","trace_propagation_headers":{"sentry-trace":"99b8cc46c48947119e3da7bcec0bff79-89de2d60915a48bf","baggage":"sentry-trace_id=99b8cc46c48947119e3da7bcec0bff79,sentry-release=Unknown"},"correlation_id":"01M77H3GRVZ8VAJ57BKTCPH8ZB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:30.481Z","job_size_bytes":3,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-30d72af0468ae640c27c6074: done: 0.02757 sec","job_status":"done","queue_duration_s":0.048785,"scheduling_latency_s":0.048799,"redis_calls":7,"redis_duration_s":0.002161,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001882,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000279,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02757,"completed_at":"2026-11-12T09:23:30.557Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004458,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2026gitlab # [ 255.771909] gitaly[899]: time="2026-11-12T09:23:30.304Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GRVZ8VAJ57BKTCPH8ZB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-12T09:23:40.038" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:30.299" grpc.time_ms=5.029 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2027gitlab # [ 255.775550] gitaly[899]: time="2026-11-12T09:23:30.330Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H3GRVZ8VAJ57BKTCPH8ZB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:23:30.329" grpc.time_ms=0.274 pid=899 span.kind=server system=grpc2028gitlab # [ 255.814214] nixbot[2660]: {"ts": "2026-11-12T09:23:30.770Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:47234 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 HTTP/1.1\" 200"}2029gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses, in 0.09 seconds)2030gitlab # [ 255.814821] gitaly[899]: time="2026-11-12T09:23:30.350Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M77H3GRVZ8VAJ57BKTCPH8ZB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-12T09:23:30.349" grpc.time_ms=0.361 pid=899 span.kind=server system=grpc2031gitlab # [ 255.817985] gitaly[899]: time="2026-11-12T09:23:30.358Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GRVZ8VAJ57BKTCPH8ZB diskcache=3a116853-84e4-493a-8742-670acdf50cec grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-12T09:23:40.046" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:23:30.306" pid=899 ref=refs/keep-around/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 remote_ip="2001:db8:1::1" revision=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 span.kind=server system=grpc username=root2032gitlab # [ 255.820277] gitaly[899]: time="2026-11-12T09:23:30.358Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=43 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5138 command.oublock=8 command.real_time_ms=45 command.spawn_token_fork_ms=0 command.system_time_ms=23 command.user_time_ms=20 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GRVZ8VAJ57BKTCPH8ZB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-12T09:23:40.046" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-12T09:23:30.306" grpc.time_ms=52.317 pid=899 ref=refs/keep-around/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 remote_ip="2001:db8:1::1" revision=e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 span.kind=server system=grpc username=root2033gitlab # [ 255.821095] gitaly[899]: time="2026-11-12T09:23:30.362Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GRVZ8VAJ57BKTCPH8ZB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:24:00.161" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:30.361" grpc.time_ms=0.305 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2034gitlab # [ 255.860152] gitaly[899]: time="2026-11-12T09:23:30.492Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3GRVZ8VAJ57BKTCPH8ZB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-12T09:24:00.491" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:30.491" grpc.time_ms=0.717 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2035gitlab # [ 255.860815] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 692036gitlab # [ 255.861470] gitaly[899]: time="2026-11-12T09:23:30.521Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3GRVZ8VAJ57BKTCPH8ZB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:24:00.519" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:30.520" grpc.time_ms=0.77 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2037gitlab # [ 255.862088] gitaly[899]: time="2026-11-12T09:23:30.838Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3HB60V1E87P10P5RGSBY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:24:00.436" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:30.837" grpc.time_ms=0.641 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2038gitlab # [ 255.865540] nix-daemon[2825]: accepted connection from pid 3696, user nixbot2039gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses2040gitlab # [ 256.941792] gitaly[899]: time="2026-11-12T09:23:31.975Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3JDYN82RX9NG4GQR9CJ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:24:01.873" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:31.973" grpc.time_ms=0.982 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2041gitlab # [ 256.983832] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1142042gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses, in 0.16 seconds)2043gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses2044gitlab # [ 258.087676] gitaly[899]: time="2026-11-12T09:23:33.120Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3KJ5VCAPNZ8XPBZZ87JW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:24:03.019" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:33.119" grpc.time_ms=0.888 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2045gitlab # [ 258.138514] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:33 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1102046gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses, in 0.15 seconds)2047gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses2048gitlab # [ 259.278845] gitaly[899]: time="2026-11-12T09:23:34.312Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3MP7FK1J4K126HQKA89D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:24:04.110" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:34.310" grpc.time_ms=1.01 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2049gitlab # [ 259.313270] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1302050gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses, in 0.17 seconds)2051gitlab # [ 259.947775] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:34.980Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"a0e81cea88d13f8710d1af16","created_at":"2026-11-12T09:23:34.962Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2946e41f79a5178e814899dbdd149335","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:34.977Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-a0e81cea88d13f8710d1af16: start","job_status":"start","queue_duration_s":0.003079,"scheduling_latency_s":0.003134}2052gitlab # [ 260.038255] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.072Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2053gitlab # [ 260.060922] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.094Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"a0e81cea88d13f8710d1af16","created_at":"2026-11-12T09:23:34.962Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2946e41f79a5178e814899dbdd149335","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:34.977Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-a0e81cea88d13f8710d1af16: done: 0.114108 sec","job_status":"done","queue_duration_s":0.003079,"scheduling_latency_s":0.003134,"redis_calls":9,"redis_duration_s":0.037236,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002901,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.031548,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002787,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.114108,"completed_at":"2026-11-12T09:23:35.094Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002335,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2054gitlab # [ 260.106656] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.101Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"01741fcdab6eb5529055b9f7","created_at":"2026-11-12T09:23:35.013Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49e00b87a8eb681be6ded164c1a5396f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.021Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-01741fcdab6eb5529055b9f7: start","job_status":"start","queue_duration_s":0.079035,"scheduling_latency_s":0.079076}2055gitlab # [ 260.248516] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.281Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"01741fcdab6eb5529055b9f7","created_at":"2026-11-12T09:23:35.013Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"49e00b87a8eb681be6ded164c1a5396f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.021Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-01741fcdab6eb5529055b9f7: done: 0.180767 sec","job_status":"done","queue_duration_s":0.079035,"scheduling_latency_s":0.079076,"redis_calls":18,"redis_duration_s":0.041382,"redis_read_bytes":19,"redis_write_bytes":4698,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.024139,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000669,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.01453,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002044,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.180767,"completed_at":"2026-11-12T09:23:35.281Z","load_balancing_strategy":"primary","db_duration_s":0.006794,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2056gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses2057gitlab # [ 260.326875] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.360Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"820b21e975adc6dabe26fd6e","created_at":"2026-11-12T09:23:35.063Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fc6086ddd3cde5021fa67860176c547","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.080Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-820b21e975adc6dabe26fd6e: start","job_status":"start","queue_duration_s":0.280201,"scheduling_latency_s":0.280258}2058gitlab # [ 260.432443] gitaly[899]: time="2026-11-12T09:23:35.464Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3NV0EYKXW2TVV1PZ8T4Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:24:05.063" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:35.463" grpc.time_ms=0.823 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2059gitlab # [ 260.448750] nix-daemon[2825]: accepted connection from pid 3732, user nixbot2060gitlab # [ 260.449994] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 912061gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses, in 0.13 seconds)2062gitlab # [ 260.460751] nix-daemon[2825]: accepted connection from pid 3696, user nixbot2063gitlab # [ 260.516740] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.550Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"820b21e975adc6dabe26fd6e","created_at":"2026-11-12T09:23:35.063Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fc6086ddd3cde5021fa67860176c547","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.080Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-820b21e975adc6dabe26fd6e: done: 0.190199 sec","job_status":"done","queue_duration_s":0.280201,"scheduling_latency_s":0.280258,"redis_calls":9,"redis_duration_s":0.020596,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00579,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012717,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002089,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.095611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.190199,"completed_at":"2026-11-12T09:23:35.550Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018217,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2064gitlab # [ 260.557968] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.591Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"7124b812b020910d8e094f8d","created_at":"2026-11-12T09:23:35.095Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"76dc4bf11ecf495ca516a8b5e264e87d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.139Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-7124b812b020910d8e094f8d: start","job_status":"start","queue_duration_s":0.451506,"scheduling_latency_s":0.45154}2065gitlab # [ 260.577585] gitaly[899]: time="2026-11-12T09:23:35.611Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3NZBVBHNP0BK8FSQCJ8K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:24:05.210" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:35.611" grpc.time_ms=0.57 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2066gitlab # [ 260.609575] gitaly[899]: time="2026-11-12T09:23:35.643Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3NZBVBHNP0BK8FSQCJ8K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:45.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:35.637" grpc.time_ms=5.788 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2067gitlab # [ 260.613261] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.647Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"7124b812b020910d8e094f8d","created_at":"2026-11-12T09:23:35.095Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"76dc4bf11ecf495ca516a8b5e264e87d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.139Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-7124b812b020910d8e094f8d: done: 0.056159 sec","job_status":"done","queue_duration_s":0.451506,"scheduling_latency_s":0.45154,"redis_calls":11,"redis_duration_s":0.011251,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003805,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005229,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002217,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000636952,"exclusive_lock_hold_duration_s":0.009908801,"duration_s":0.056159,"completed_at":"2026-11-12T09:23:35.647Z","load_balancing_strategy":"primary","db_duration_s":0.010481,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2068gitlab # [ 260.619534] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.651Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::PipelineArtifacts::ExpireArtifactsWorker","jid":"6a5ebb6bec6400c0a5b2e99e","created_at":"2026-11-12T09:23:35.183Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1cfccc113e06869c084ade2d56e8a0f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:35dc78c13bdf78a98c09fc2795030a4961be87ad882288b77a6a3c9be677e4d5","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.202Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::ExpireArtifactsWorker JID-6a5ebb6bec6400c0a5b2e99e: start","job_status":"start","queue_duration_s":0.448861,"scheduling_latency_s":0.448882}2069gitlab # [ 260.644340] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:35 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1462070gitlab # [ 260.673256] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.707Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::PipelineArtifacts::ExpireArtifactsWorker","jid":"6a5ebb6bec6400c0a5b2e99e","created_at":"2026-11-12T09:23:35.183Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1cfccc113e06869c084ade2d56e8a0f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:35dc78c13bdf78a98c09fc2795030a4961be87ad882288b77a6a3c9be677e4d5","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.202Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::ExpireArtifactsWorker JID-6a5ebb6bec6400c0a5b2e99e: done: 0.055516 sec","job_status":"done","queue_duration_s":0.448861,"scheduling_latency_s":0.448882,"redis_calls":12,"redis_duration_s":0.014359,"redis_read_bytes":11,"redis_write_bytes":1481,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002932,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005269,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006158,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":548,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.004772953,"exclusive_lock_hold_duration_s":0.008745805,"extra.ci_pipeline_artifacts_expire_artifacts_worker.destroyed_pipeline_artifacts_count":0,"duration_s":0.055516,"completed_at":"2026-11-12T09:23:35.707Z","load_balancing_strategy":"primary","db_duration_s":0.020744,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2071gitlab # [ 260.674507] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.708Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"014b74ab98be3a53f788b351","created_at":"2026-11-12T09:23:35.233Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c6bbdb66ec92cc93dbc53ba6f6f4a56","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.250Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-014b74ab98be3a53f788b351: start","job_status":"start","queue_duration_s":0.457753,"scheduling_latency_s":0.457771}2072gitlab # [ 260.685413] gitaly[899]: time="2026-11-12T09:23:35.718Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3P4401VT305ZNH2MVPSQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:24:05.316" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:35.717" grpc.time_ms=0.803 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2073gitlab # [ 260.732870] gitaly[899]: time="2026-11-12T09:23:35.744Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3P4401VT305ZNH2MVPSQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:45.107" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:35.737" grpc.time_ms=7.016 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2074gitlab # [ 260.733705] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.756Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"014b74ab98be3a53f788b351","created_at":"2026-11-12T09:23:35.233Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c6bbdb66ec92cc93dbc53ba6f6f4a56","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.250Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-014b74ab98be3a53f788b351: done: 0.048035 sec","job_status":"done","queue_duration_s":0.457753,"scheduling_latency_s":0.457771,"redis_calls":12,"redis_duration_s":0.002337,"redis_read_bytes":11,"redis_write_bytes":1281,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000885,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000823,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000629,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":476,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000342223,"exclusive_lock_hold_duration_s":0.01581458,"duration_s":0.048035,"completed_at":"2026-11-12T09:23:35.756Z","load_balancing_strategy":"primary","db_duration_s":0.012894,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2075gitlab # [ 260.780644] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.758Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3226e2d5674b2a2388d9c5b8","created_at":"2026-11-12T09:23:35.205Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-fae5697761a14fb7","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"49e00b87a8eb681be6ded164c1a5396f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.253Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3226e2d5674b2a2388d9c5b8: start","job_status":"start","queue_duration_s":0.504965,"scheduling_latency_s":0.504983}2076gitlab # [ 260.781543] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.805Z","message":"No existing merge request to be cleaned up."}2077gitlab # [ 260.782074] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.815Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3226e2d5674b2a2388d9c5b8","created_at":"2026-11-12T09:23:35.205Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-fae5697761a14fb7","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"49e00b87a8eb681be6ded164c1a5396f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.253Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3226e2d5674b2a2388d9c5b8: done: 0.057175 sec","job_status":"done","queue_duration_s":0.504965,"scheduling_latency_s":0.504983,"redis_calls":12,"redis_duration_s":0.003539,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000298,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002148,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.057175,"completed_at":"2026-11-12T09:23:35.815Z","load_balancing_strategy":"primary","db_duration_s":0.037108,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2078gitlab # [ 260.795875] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:35 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 962079gitlab # [ 260.798172] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.816Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e9f4d512ce0478f404bfd3d5","created_at":"2026-11-12T09:23:35.205Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-fae5697761a14fb7","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"49e00b87a8eb681be6ded164c1a5396f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.253Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e9f4d512ce0478f404bfd3d5: start","job_status":"start","queue_duration_s":0.563142,"scheduling_latency_s":0.563158}2080gitlab # [ 260.846854] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.826Z","message":"No existing merge request to be cleaned up."}2081gitlab # [ 260.847566] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.831Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e9f4d512ce0478f404bfd3d5","created_at":"2026-11-12T09:23:35.205Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-fae5697761a14fb7","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"49e00b87a8eb681be6ded164c1a5396f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.253Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e9f4d512ce0478f404bfd3d5: done: 0.015374 sec","job_status":"done","queue_duration_s":0.563142,"scheduling_latency_s":0.563158,"redis_calls":10,"redis_duration_s":0.0013570000000000001,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000814,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015374,"completed_at":"2026-11-12T09:23:35.831Z","load_balancing_strategy":"primary","db_duration_s":0.005439,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2082gitlab # [ 260.848739] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.832Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"62151f74f0049206cc0f6a7c","created_at":"2026-11-12T09:23:35.205Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-fae5697761a14fb7","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"49e00b87a8eb681be6ded164c1a5396f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.253Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-62151f74f0049206cc0f6a7c: start","job_status":"start","queue_duration_s":0.579397,"scheduling_latency_s":0.579413}2083gitlab # [ 260.858764] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.841Z","message":"No existing merge request to be cleaned up."}2084gitlab # [ 260.860196] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.846Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"62151f74f0049206cc0f6a7c","created_at":"2026-11-12T09:23:35.205Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-fae5697761a14fb7","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"49e00b87a8eb681be6ded164c1a5396f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.253Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-62151f74f0049206cc0f6a7c: done: 0.013188 sec","job_status":"done","queue_duration_s":0.579397,"scheduling_latency_s":0.579413,"redis_calls":10,"redis_duration_s":0.001418,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000862,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013188,"completed_at":"2026-11-12T09:23:35.845Z","load_balancing_strategy":"primary","db_duration_s":0.001137,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2085gitlab # [ 260.901587] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.846Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7f22152bbd3f4ae2fdf43f33","created_at":"2026-11-12T09:23:35.205Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-fae5697761a14fb7","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"49e00b87a8eb681be6ded164c1a5396f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.253Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7f22152bbd3f4ae2fdf43f33: start","job_status":"start","queue_duration_s":0.593448,"scheduling_latency_s":0.593462}2086gitlab # [ 260.942134] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.854Z","message":"No existing merge request to be cleaned up."}2087gitlab # [ 260.942899] gitaly[899]: time="2026-11-12T09:23:35.837Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3P7J1X3QBY9G80M9T16F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-12T09:24:05.436" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-12T09:23:35.836" grpc.time_ms=0.543 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2088gitlab # [ 260.943577] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:35 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1082089gitlab # [ 260.947406] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.859Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7f22152bbd3f4ae2fdf43f33","created_at":"2026-11-12T09:23:35.205Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-fae5697761a14fb7","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"49e00b87a8eb681be6ded164c1a5396f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.253Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7f22152bbd3f4ae2fdf43f33: done: 0.012978 sec","job_status":"done","queue_duration_s":0.593448,"scheduling_latency_s":0.593462,"redis_calls":10,"redis_duration_s":0.001386,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000828,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012978,"completed_at":"2026-11-12T09:23:35.859Z","load_balancing_strategy":"primary","db_duration_s":0.002014,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2090gitlab # [ 260.992578] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:35.860Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"9645f46055a77c7606035c62","created_at":"2026-11-12T09:23:35.268Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7ac3575689c742587ebf190667b4b09","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.359Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-9645f46055a77c7606035c62: start","job_status":"start","queue_duration_s":0.50132,"scheduling_latency_s":0.501333}2091gitlab # [ 260.993466] gitaly[899]: time="2026-11-12T09:23:35.863Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M77H3P7J1X3QBY9G80M9T16F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-12T09:23:45.228" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-12T09:23:35.858" grpc.time_ms=5.243 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2092gitlab # [ 261.033497] nixbot[2660]: {"ts": "2026-11-12T09:23:36.067Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 2, "sha": "e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []}2093gitlab # [ 261.105380] gitaly[899]: time="2026-11-12T09:23:36.139Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3PH3Z91FKYN61G1EZ5BV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:36.133" grpc.time_ms=6.103 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc2094gitlab # [ 261.108652] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:36 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 402095gitlab # [ 261.128087] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.161Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"9645f46055a77c7606035c62","created_at":"2026-11-12T09:23:35.268Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7ac3575689c742587ebf190667b4b09","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.359Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-9645f46055a77c7606035c62: done: 0.301144 sec","job_status":"done","queue_duration_s":0.50132,"scheduling_latency_s":0.501333,"redis_calls":564,"redis_duration_s":0.113127,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.013662,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.09921,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.200858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.301144,"completed_at":"2026-11-12T09:23:36.161Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2096gitlab # [ 261.148886] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.163Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"cc7e45948c3905ff7c9e05a8","created_at":"2026-11-12T09:23:35.403Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"709217039c79956523c8da8d301f9b58","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.418Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-cc7e45948c3905ff7c9e05a8: start","job_status":"start","queue_duration_s":0.7452,"scheduling_latency_s":0.745226}2097gitlab # [ 261.149687] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.180Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"cc7e45948c3905ff7c9e05a8","created_at":"2026-11-12T09:23:35.403Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"709217039c79956523c8da8d301f9b58","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.418Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-cc7e45948c3905ff7c9e05a8: done: 0.017029 sec","job_status":"done","queue_duration_s":0.7452,"scheduling_latency_s":0.745226,"redis_calls":9,"redis_duration_s":0.001212,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000273,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00069,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000249,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017029,"completed_at":"2026-11-12T09:23:36.180Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001837,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2098gitlab # [ 261.198986] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.181Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"7686fc488954e6b7472bb774","created_at":"2026-11-12T09:23:35.432Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6c9951aa02abc7dc1105ad5a2eb9946","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-12T09:23:35.523Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-7686fc488954e6b7472bb774: start","job_status":"start","queue_duration_s":0.658105,"scheduling_latency_s":0.65812}2099gitlab # [ 261.239995] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.273Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2100gitlab # [ 261.249379] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.283Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"7686fc488954e6b7472bb774","created_at":"2026-11-12T09:23:35.432Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6c9951aa02abc7dc1105ad5a2eb9946","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-12T09:23:35.523Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-7686fc488954e6b7472bb774: done: 0.10195 sec","job_status":"done","queue_duration_s":0.658105,"scheduling_latency_s":0.65812,"redis_calls":9,"redis_duration_s":0.079815,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000593,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001192,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.07803,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017491,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.10195,"completed_at":"2026-11-12T09:23:36.283Z","load_balancing_strategy":"primary","db_duration_s":0.001177,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2101gitlab # [ 261.251866] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.284Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"3ef3ae7c92d6472d2885641f","created_at":"2026-11-12T09:23:35.528Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14e5e8aee425a844384236c4327e230c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.535Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-3ef3ae7c92d6472d2885641f: start","job_status":"start","queue_duration_s":0.74915,"scheduling_latency_s":0.749173}2102gitlab # [ 261.261939] gitaly[899]: time="2026-11-12T09:23:36.293Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M77H3PJGSYX0591GBQ78TVCX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:36.272" pid=899 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2103gitlab # [ 261.270957] gitlab-workhorse[825]: time="2026-11-12T09:23:36Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672104gitlab # [ 261.272760] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:36 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1512105gitlab # [ 261.274438] gitaly[899]: time="2026-11-12T09:23:36.294Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3PJGSYX0591GBQ78TVCX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:36.272" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc2106gitlab # [ 261.320648] gitaly[899]: time="2026-11-12T09:23:36.294Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3PJGSYX0591GBQ78TVCX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:36.272" grpc.time_ms=21.613 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc2107gitlab # [ 261.321669] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.330Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"3ef3ae7c92d6472d2885641f","created_at":"2026-11-12T09:23:35.528Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14e5e8aee425a844384236c4327e230c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.535Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-3ef3ae7c92d6472d2885641f: done: 0.046096 sec","job_status":"done","queue_duration_s":0.74915,"scheduling_latency_s":0.749173,"redis_calls":13,"redis_duration_s":0.0020109999999999998,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000541,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000745,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000725,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000390273,"exclusive_lock_hold_duration_s":0.028266163,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.046096,"completed_at":"2026-11-12T09:23:36.330Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012436,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2108gitlab # [ 261.374632] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.331Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"434317f5f25ccecc0baf1aab","created_at":"2026-11-12T09:23:35.543Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0a8206dd1d3e1f8c744e61ce10061c0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.590Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-434317f5f25ccecc0baf1aab: start","job_status":"start","queue_duration_s":0.741384,"scheduling_latency_s":0.741399}2109gitlab # [ 261.377127] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.339Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2110gitlab # [ 261.377796] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.344Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"434317f5f25ccecc0baf1aab","created_at":"2026-11-12T09:23:35.543Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0a8206dd1d3e1f8c744e61ce10061c0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.590Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-434317f5f25ccecc0baf1aab: done: 0.012405 sec","job_status":"done","queue_duration_s":0.741384,"scheduling_latency_s":0.741399,"redis_calls":9,"redis_duration_s":0.001215,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000271,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000673,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012405,"completed_at":"2026-11-12T09:23:36.344Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000309,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2111gitlab # [ 261.397699] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.345Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ae1f7380d69e7c52fc679be5","created_at":"2026-11-12T09:23:35.600Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6bca83618cb058b098f2e3f9cb1e5ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.609Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ae1f7380d69e7c52fc679be5: start","job_status":"start","queue_duration_s":0.735643,"scheduling_latency_s":0.735675}2112gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses2113gitlab # [ 261.443965] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.421Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ae1f7380d69e7c52fc679be5","created_at":"2026-11-12T09:23:35.600Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6bca83618cb058b098f2e3f9cb1e5ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.609Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ae1f7380d69e7c52fc679be5: done: 0.075947 sec","job_status":"done","queue_duration_s":0.735643,"scheduling_latency_s":0.735675,"redis_calls":70,"redis_duration_s":0.010733999999999999,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001456,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000478,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.00836,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00044,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.075947,"completed_at":"2026-11-12T09:23:36.421Z","load_balancing_strategy":"primary","db_duration_s":0.004336,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2114gitlab # [ 261.446795] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.422Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b711d55671607be10b479af3","created_at":"2026-11-12T09:23:35.614Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"841f8ac9669cfe5b192d921b7420ab4f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.623Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b711d55671607be10b479af3: start","job_status":"start","queue_duration_s":0.798896,"scheduling_latency_s":0.798917}2115gitlab # [ 261.460875] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:36 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 462116gitlab # [ 261.462428] gitaly[899]: time="2026-11-12T09:23:36.435Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M77H3PT5MPZKD3EYT1FN8AAG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:36.429" grpc.time_ms=6.558 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc2117gitlab # [ 261.473442] gitaly[899]: time="2026-11-12T09:23:36.494Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M77H3PVPR92YBYXBDFSHBEXJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:36.484" pid=899 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc2118gitlab # [ 261.483033] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.447Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b711d55671607be10b479af3","created_at":"2026-11-12T09:23:35.614Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"841f8ac9669cfe5b192d921b7420ab4f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.623Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b711d55671607be10b479af3: done: 0.025068 sec","job_status":"done","queue_duration_s":0.798896,"scheduling_latency_s":0.798917,"redis_calls":11,"redis_duration_s":0.001861,"redis_read_bytes":9,"redis_write_bytes":1386,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000736,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000817,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000308,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025068,"completed_at":"2026-11-12T09:23:36.447Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00149,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2119gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7/statuses, in 0.12 seconds)2120(finished: subtest: eval and build post commit statuses, in 5.84 seconds)2121(finished: run the VM test script, in 317.07 seconds)2122gitlab # [ 261.540797] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.448Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"71e1d42cd515d1b67df1c4e8","created_at":"2026-11-12T09:23:35.630Z","trace_propagation_headers":{"sentry-trace":"496795e6486648f49e63e07ac36e1665-0178bd0018234974","baggage":"sentry-trace_id=496795e6486648f49e63e07ac36e1665,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1598d9d8082c8ecd9bc49f468dd52aef","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-12T09:23:35.646Z","job_size_bytes":2,"pid":2534,"sidekiq_tid":"m7wu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-71e1d42cd515d1b67df1c4e8: start","job_status":"start","queue_duration_s":0.802784,"scheduling_latency_s":0.802804}2123gitlab # [ 261.541724] gitlab-workhorse[825]: time="2026-11-12T09:23:36Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672124gitlab # [ 261.544084] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/11/12:09:23:36 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 572125gitlab # [ 261.544652] gitaly[899]: time="2026-11-12T09:23:36.494Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M77H3PVPR92YBYXBDFSHBEXJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-12T09:23:36.484" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=899 remote_ip="2001:db8:1::1" span.kind=server system=grpc2126test script finished in 317.09s2127cleanup2128kill QemuMachine (pid 46)2129gitlab # [ 261.551864] sidekiq-cluster[2534]: {"severity":"INFO","time":"2026-11-12T09:23:36.472Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncqemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14)2130gitlab # E2131(finished: cleanup, in 1.51 seconds)2132additionally exposed symbols:2133 gitlab,2134 vlan1,2135 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2136[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-11-12T09:22:38.510Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2137[]2138[]2139[]2140[]2141[]2142[]2143[]2144[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-11-12T09:23:30.196699Z', 'started_at': '2026-11-12T09:23:30.441350Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}]2145[{'id': 5, 'sha': 'e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-12T09:23:30.412Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2146[{'id': 5, 'sha': 'e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-12T09:23:30.412Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2147[{'id': 5, 'sha': 'e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-12T09:23:30.412Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2148[{'id': 5, 'sha': 'e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-12T09:23:30.412Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2149[{'id': 5, 'sha': 'e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-12T09:23:30.412Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2150[{'id': 5, 'sha': 'e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 4s (1 warning)', 'created_at': '2026-11-12T09:23:30.412Z', 'started_at': None, 'finished_at': '2026-11-12T09:23:35.660Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'e26299576e5f50eaf1e806b8c6a8eeacb2bbb2d7', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-11-12T09:23:35.758Z', 'started_at': None, 'finished_at': '2026-11-12T09:23:35.876Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]