vm-test-run-nixbot-gitlab
checks.x86_64-linux.nixbot-gitlab
· build #492
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab: QEMU running (pid 46)10gitlab # Disk image does not exist, creating the virtualisation disk image...11gitlab # Formatting '/build/vm-state-gitlab/tmp.hqwWSkpCbc', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15(finished: start all VMs, in 60.40 seconds)16gitlab # Filesystem UUID: 7d9b8c44-852b-4536-ae0c-a9b16979dd1c17gitlab: waiting for unit setup-gitlab.service18gitlab # Superblock backups stored on blocks:19gitlab: waiting for the VM to finish booting20gitlab # 32768, 98304, 163840, 22937621gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)31gitlab # 32gitlab # 33gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA0034gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)...35gitlab # 36gitlab # 37gitlab # 38gitlab # 39gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB0040gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)...41gitlab # 42gitlab # 43gitlab # Booting from ROM...44gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 202645gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/2w0b254hl61bswmdix102sqs162p3ip5-nixos-system-gitlab-test/init regInfo=/nix/store/j3mijxqbw97fzbcyf86rdjijya3glk2x-closure-info/registration console=ttyS0,115200n8 console=tty046gitlab # [ 0.000000] BIOS-provided physical RAM map:47gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable48gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved49gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved50gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable51gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved52gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved53gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved54gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable55gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved56gitlab # [ 0.000000] NX (Execute Disable) protection: active57gitlab # [ 0.000000] APIC: Static calls initialized58gitlab # [ 0.000000] SMBIOS 2.8 present.59gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/201460gitlab # [ 0.000000] DMI: Memory slots populated: 1/161gitlab # [ 0.000000] Hypervisor detected: KVM62gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000063gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d0064gitlab # [ 0.000000] kvm-clock: using sched offset of 479301807 cycles65gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns66gitlab # [ 0.000005] tsc: Detected 2400.010 MHz processor67gitlab # [ 0.000805] last_pfn = 0x1c0000 max_arch_pfn = 0x1000000000068gitlab # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs69gitlab # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT70gitlab # [ 0.000878] last_pfn = 0xbffd8 max_arch_pfn = 0x1000000000071gitlab # [ 0.002731] found SMP MP-table at [mem 0x000f5470-0x000f547f]72gitlab # [ 0.002742] Using GB pages for direct mapping73gitlab # [ 0.002786] RAMDISK: [mem 0xbe31c000-0xbffcffff]74gitlab # [ 0.002796] ACPI: Early table checksum verification disabled75gitlab # [ 0.002799] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )76gitlab # [ 0.002802] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)77gitlab # [ 0.002806] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)78gitlab # [ 0.002812] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001)79gitlab # [ 0.002815] ACPI: FACS 0x00000000BFFE0000 00004080gitlab # [ 0.002816] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001)81gitlab # [ 0.002818] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)82gitlab # [ 0.002826] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)83gitlab # [ 0.002828] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2]84gitlab # [ 0.002829] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e]85gitlab # [ 0.002829] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f]86gitlab # [ 0.002830] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482]87gitlab # [ 0.002830] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba]88gitlab # [ 0.002831] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2]89gitlab # [ 0.003061] No NUMA configuration found90gitlab # [ 0.003062] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff]91gitlab # [ 0.003065] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff]92gitlab # [ 0.005426] Zone ranges:93gitlab # [ 0.005427] DMA [mem 0x0000000000001000-0x0000000000ffffff]94gitlab # [ 0.005429] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]95gitlab # [ 0.005430] Normal [mem 0x0000000100000000-0x00000001bfffffff]96gitlab # [ 0.005431] Device empty97gitlab # [ 0.005432] Movable zone start for each node98gitlab # [ 0.005432] Early memory node ranges99gitlab # [ 0.005433] node 0: [mem 0x0000000000001000-0x000000000009efff]100gitlab # [ 0.005434] node 0: [mem 0x0000000000100000-0x00000000bffd7fff]101gitlab # [ 0.005435] node 0: [mem 0x0000000100000000-0x00000001bfffffff]102gitlab # [ 0.005436] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff]103gitlab # [ 0.005457] On node 0, zone DMA: 1 pages in unavailable ranges104gitlab # [ 0.005731] On node 0, zone DMA: 97 pages in unavailable ranges105gitlab # [ 0.118324] On node 0, zone Normal: 40 pages in unavailable ranges106gitlab # [ 0.118740] ACPI: PM-Timer IO Port: 0x608107gitlab # [ 0.118755] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])108gitlab # [ 0.118791] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23109gitlab # [ 0.118794] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)110gitlab # [ 0.118796] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)111gitlab # [ 0.118797] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)112gitlab # [ 0.118798] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)113gitlab # [ 0.118799] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)114gitlab # [ 0.118801] ACPI: Using ACPI (MADT) for SMP configuration information115gitlab # [ 0.118802] ACPI: HPET id: 0x8086a201 base: 0xfed00000116gitlab # [ 0.118807] TSC deadline timer available117gitlab # [ 0.118812] CPU topo: Max. logical packages: 1118gitlab # [ 0.118813] CPU topo: Max. logical dies: 1119gitlab # [ 0.118813] CPU topo: Max. dies per package: 1120gitlab # [ 0.118816] CPU topo: Max. threads per core: 1121gitlab # [ 0.118817] CPU topo: Num. cores per package: 4122gitlab # [ 0.118818] CPU topo: Num. threads per package: 4123gitlab # [ 0.118824] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs124gitlab # [ 0.118848] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()125gitlab # [ 0.118864] kvm-guest: KVM setup pv remote TLB flush126gitlab # [ 0.118867] kvm-guest: setup PV sched yield127gitlab # [ 0.118883] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]128gitlab # [ 0.118885] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]129gitlab # [ 0.118886] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff]130gitlab # [ 0.118887] [mem 0xc0000000-0xfeffbfff] available for PCI devices131gitlab # [ 0.118889] Booting paravirtualized kernel on KVM132gitlab # [ 0.118892] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns133gitlab # [ 0.123359] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1134gitlab # [ 0.125998] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288135gitlab # [ 0.126050] kvm-guest: PV spinlocks enabled136gitlab # [ 0.126052] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)137gitlab # [ 0.126055] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/2w0b254hl61bswmdix102sqs162p3ip5-nixos-system-gitlab-test/init regInfo=/nix/store/j3mijxqbw97fzbcyf86rdjijya3glk2x-closure-info/registration console=ttyS0,115200n8 console=tty0138gitlab # [ 0.126149] Unknown kernel command line parameters "regInfo=/nix/store/j3mijxqbw97fzbcyf86rdjijya3glk2x-closure-info/registration", will be passed to user space.139gitlab # [ 0.126162] random: crng init done140gitlab # [ 0.126163] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes141gitlab # [ 0.135391] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)142gitlab # [ 0.139856] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)143gitlab # [ 0.139931] software IO TLB: area num 4.144gitlab # [ 0.216719] Fallback order for Node 0: 0145gitlab # [ 0.216726] Built 1 zonelists, mobility grouping on. Total pages: 1572726146gitlab # [ 0.216728] Policy zone: Normal147gitlab # [ 0.219696] mem auto-init: stack:all(zero), heap alloc:on, heap free:off148gitlab # [ 0.229315] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1149gitlab # [ 0.243215] allocated 12582912 bytes of page_ext150gitlab # [ 0.253261] ftrace: allocating 48717 entries in 192 pages151gitlab # [ 0.253263] ftrace: allocated 192 pages with 2 groups152gitlab # [ 0.254548] Dynamic Preempt: lazy153gitlab # [ 0.254728] rcu: Preemptible hierarchical RCU implementation.154gitlab # [ 0.254728] rcu: RCU event tracing is enabled.155gitlab # [ 0.254729] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.156gitlab # [ 0.254731] Trampoline variant of Tasks RCU enabled.157gitlab # [ 0.254731] Rude variant of Tasks RCU enabled.158gitlab # [ 0.254732] Tracing variant of Tasks RCU enabled.159gitlab # [ 0.254733] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.160gitlab # [ 0.254733] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4161gitlab # [ 0.254754] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.162gitlab # [ 0.254756] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.163gitlab # [ 0.254757] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.164gitlab # [ 0.259194] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16165gitlab # [ 0.259497] rcu: srcu_init: Setting srcu_struct sizes based on contention.166gitlab # [ 0.259506] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns167gitlab # [ 0.259613] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)168gitlab # [ 0.263211] Console: colour VGA+ 80x25169gitlab # [ 0.263214] printk: legacy console [tty0] enabled170gitlab # [ 0.294130] printk: legacy console [ttyS0] enabled171gitlab # [ 0.405201] ACPI: Core revision 20250807172gitlab # [ 0.406143] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns173gitlab # [ 0.407767] APIC: Switch to symmetric I/O mode setup174gitlab # [ 0.408856] x2apic enabled175gitlab # [ 0.409615] APIC: Switched APIC routing to: physical x2apic176gitlab # [ 0.410580] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask()177gitlab # [ 0.411872] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself()178gitlab # [ 0.413366] kvm-guest: setup PV IPIs179gitlab # [ 0.414976] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1180gitlab # [ 0.416055] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns181gitlab # [ 0.417879] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010)182gitlab # [ 0.418963] x86/cpu: User Mode Instruction Prevention (UMIP) activated183gitlab # [ 0.420007] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127184gitlab # [ 0.421876] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0185gitlab # [ 0.422880] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto186gitlab # [ 0.424876] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl187gitlab # [ 0.425876] Transient Scheduler Attacks: Vulnerable: No microcode188gitlab # [ 0.427876] Spectre V2 : Mitigation: Enhanced / Automatic IBRS189gitlab # [ 0.428875] Speculative Return Stack Overflow: Mitigation: Safe RET190gitlab # [ 0.429876] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization191gitlab # [ 0.430884] Spectre V2 : Enabling IBPB for BPF192gitlab # [ 0.431876] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier193gitlab # [ 0.432877] active return thunk: srso_alias_return_thunk194gitlab # [ 0.434903] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'195gitlab # [ 0.435876] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'196gitlab # [ 0.436876] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'197gitlab # [ 0.437876] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'198gitlab # [ 0.438876] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'199gitlab # [ 0.439876] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'200gitlab # [ 0.441876] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'201gitlab # [ 0.442876] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256202gitlab # [ 0.443876] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64203gitlab # [ 0.444876] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512204gitlab # [ 0.445876] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024205gitlab # [ 0.447876] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8206gitlab # [ 0.448876] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.207gitlab # [ 0.479315] Freeing SMP alternatives memory: 44K208gitlab # [ 0.479879] pid_max: default: 32768 minimum: 301209gitlab # [ 0.480986] LSM: initializing lsm=capability,landlock,yama,bpf,ima210gitlab # [ 0.481972] landlock: Up and running.211gitlab # [ 0.482877] Yama: becoming mindful.212gitlab # [ 0.483727] LSM support for eBPF active213gitlab # [ 0.484843] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)214gitlab # [ 0.486028] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)215gitlab # [ 0.488980] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)216gitlab # [ 0.490514] Performance Events: Fam17h+ core perfctr, AMD PMU driver.217gitlab # [ 0.490885] ... version: 2218gitlab # [ 0.491672] ... bit width: 48219gitlab # [ 0.491878] ... generic counters: 6220gitlab # [ 0.492630] ... generic bitmap: 000000000000003f221gitlab # [ 0.492878] ... fixed-purpose counters: 0222gitlab # [ 0.493661] ... fixed-purpose bitmap: 0000000000000000223gitlab # [ 0.493878] ... value mask: 0000ffffffffffff224gitlab # [ 0.494844] ... max period: 00007fffffffffff225gitlab # [ 0.495648] ... global_ctrl mask: 000000000000003f226gitlab # [ 0.495998] signal: max sigframe size: 3376227gitlab # [ 0.496859] rcu: Hierarchical SRCU implementation.228gitlab # [ 0.497553] rcu: Max phase no-delay instances is 400.229gitlab # [ 0.498075] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level230gitlab # [ 0.503400] smp: Bringing up secondary CPUs ...231gitlab # [ 0.504294] smpboot: x86: Booting SMP configuration:232gitlab # [ 0.504883] .... node #0, CPUs: #1 #2 #3233gitlab # [ 0.506961] smp: Brought up 1 node, 4 CPUs234gitlab # [ 0.508611] smpboot: Total of 4 processors activated (19200.08 BogoMIPS)235gitlab # [ 0.509308] Memory: 6011348K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260100K reserved, 0K cma-reserved)236gitlab # [ 0.510205] devtmpfs: initialized237gitlab # [ 0.511108] x86/mm: Memory block size: 128MB238gitlab # [ 0.514189] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)239gitlab # [ 0.516017] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).240gitlab # [ 0.516976] pinctrl core: initialized pinctrl subsystem241gitlab # [ 0.518164] PM: RTC time: 10:54:26, date: 2026-11-10242gitlab # [ 0.521883] NET: Registered PF_NETLINK/PF_ROUTE protocol family243gitlab # [ 0.524236] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations244gitlab # [ 0.524935] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations245gitlab # [ 0.527449] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations246gitlab # [ 0.527891] audit: initializing netlink subsys (disabled)247gitlab # [ 0.529064] audit: type=2000 audit(1794308066.269:1): state=initialized audit_enabled=0 res=1248gitlab # [ 0.529153] thermal_sys: Registered thermal governor 'fair_share'249gitlab # [ 0.529882] thermal_sys: Registered thermal governor 'bang_bang'250gitlab # [ 0.530879] thermal_sys: Registered thermal governor 'step_wise'251gitlab # [ 0.531878] thermal_sys: Registered thermal governor 'user_space'252gitlab # [ 0.532878] thermal_sys: Registered thermal governor 'power_allocator'253gitlab # [ 0.533893] cpuidle: using governor menu254gitlab # [ 0.536082] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5255gitlab # [ 0.537109] PCI: Using configuration type 1 for base access256gitlab # [ 0.537880] PCI: Using configuration type 1 for extended access257gitlab # [ 0.539040] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.258gitlab # [ 0.540891] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages259gitlab # [ 0.541879] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page260gitlab # [ 0.542880] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages261gitlab # [ 0.543878] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page262gitlab # [ 0.547103] ACPI: Added _OSI(Module Device)263gitlab # [ 0.547880] ACPI: Added _OSI(Processor Device)264gitlab # [ 0.548645] ACPI: Added _OSI(Processor Aggregator Device)265gitlab # [ 0.552014] ACPI: 1 ACPI AML tables successfully acquired and loaded266gitlab # [ 0.557247] ACPI: Interpreter enabled267gitlab # [ 0.558936] ACPI: PM: (supports S0 S3 S4 S5)268gitlab # [ 0.559713] ACPI: Using IOAPIC for interrupt routing269gitlab # [ 0.560697] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug270gitlab # [ 0.561880] PCI: Using E820 reservations for host bridge windows271gitlab # [ 0.563079] ACPI: Enabled 2 GPEs in block 00 to 0F272gitlab # [ 0.568624] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])273gitlab # [ 0.569886] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]274gitlab # [ 0.571406] acpiphp: Slot [3] registered275gitlab # [ 0.572902] acpiphp: Slot [4] registered276gitlab # [ 0.573660] acpiphp: Slot [5] registered277gitlab # [ 0.573909] acpiphp: Slot [6] registered278gitlab # [ 0.574624] acpiphp: Slot [7] registered279gitlab # [ 0.575900] acpiphp: Slot [8] registered280gitlab # [ 0.576662] acpiphp: Slot [9] registered281gitlab # [ 0.576911] acpiphp: Slot [10] registered282gitlab # [ 0.577645] acpiphp: Slot [11] registered283gitlab # [ 0.578899] acpiphp: Slot [12] registered284gitlab # [ 0.579663] acpiphp: Slot [13] registered285gitlab # [ 0.579916] acpiphp: Slot [14] registered286gitlab # [ 0.580641] acpiphp: Slot [15] registered287gitlab # [ 0.581899] acpiphp: Slot [16] registered288gitlab # [ 0.582669] acpiphp: Slot [17] registered289gitlab # [ 0.582918] acpiphp: Slot [18] registered290gitlab # [ 0.583649] acpiphp: Slot [19] registered291gitlab # [ 0.584898] acpiphp: Slot [20] registered292gitlab # [ 0.585674] acpiphp: Slot [21] registered293gitlab # [ 0.585899] acpiphp: Slot [22] registered294gitlab # [ 0.586633] acpiphp: Slot [23] registered295gitlab # [ 0.587898] acpiphp: Slot [24] registered296gitlab # [ 0.588671] acpiphp: Slot [25] registered297gitlab # [ 0.588898] acpiphp: Slot [26] registered298gitlab # [ 0.589637] acpiphp: Slot [27] registered299gitlab # [ 0.590908] acpiphp: Slot [28] registered300gitlab # [ 0.591676] acpiphp: Slot [29] registered301gitlab # [ 0.591898] acpiphp: Slot [30] registered302gitlab # [ 0.592624] acpiphp: Slot [31] registered303gitlab # [ 0.593920] PCI host bridge to bus 0000:00304gitlab # [ 0.594704] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]305gitlab # [ 0.595854] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]306gitlab # [ 0.596777] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]307gitlab # [ 0.597879] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]308gitlab # [ 0.598879] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window]309gitlab # [ 0.600893] pci_bus 0000:00: root bus resource [bus 00-ff]310gitlab # [ 0.602060] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint311gitlab # [ 0.603369] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint312gitlab # [ 0.605300] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint313gitlab # [ 0.608931] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f]314gitlab # [ 0.609865] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk315gitlab # [ 0.610799] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk316gitlab # [ 0.611738] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk317gitlab # [ 0.612800] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk318gitlab # [ 0.613917] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint319gitlab # [ 0.616935] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f]320gitlab # [ 0.618831] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint321gitlab # [ 0.621142] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI322gitlab # [ 0.621920] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB323gitlab # [ 0.624377] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint324gitlab # [ 0.640971] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]325gitlab # [ 0.641894] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]326gitlab # [ 0.642902] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref]327gitlab # [ 0.644162] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]328gitlab # [ 0.646602] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint329gitlab # [ 0.651913] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]330gitlab # [ 0.652849] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]331gitlab # [ 0.653727] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref]332gitlab # [ 0.665888] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]333gitlab # [ 0.668377] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint334gitlab # [ 0.672699] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f]335gitlab # [ 0.673681] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]336gitlab # [ 0.674708] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref]337gitlab # [ 0.677156] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint338gitlab # [ 0.682044] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf]339gitlab # [ 0.682886] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]340gitlab # [ 0.683900] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref]341gitlab # [ 0.686166] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint342gitlab # [ 0.689968] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df]343gitlab # [ 0.690900] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]344gitlab # [ 0.691925] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref]345gitlab # [ 0.695813] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint346gitlab # [ 0.701926] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]347gitlab # [ 0.703912] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]348gitlab # [ 0.704901] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref]349gitlab # [ 0.707379] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint350gitlab # [ 0.711914] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff]351gitlab # [ 0.712827] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]352gitlab # [ 0.714901] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref]353gitlab # [ 0.717266] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint354gitlab # [ 0.720911] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff]355gitlab # [ 0.721887] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]356gitlab # [ 0.722906] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref]357gitlab # [ 0.724886] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref]358gitlab # [ 0.727169] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint359gitlab # [ 0.729891] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]360gitlab # [ 0.730901] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref]361gitlab # [ 0.733118] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint362gitlab # [ 0.737704] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f]363gitlab # [ 0.738610] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]364gitlab # [ 0.738901] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref]365gitlab # [ 0.742111] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint366gitlab # [ 0.746886] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f]367gitlab # [ 0.747885] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff]368gitlab # [ 0.748900] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref]369gitlab # [ 0.754632] ACPI: PCI: Interrupt link LNKA configured for IRQ 10370gitlab # [ 0.756060] ACPI: PCI: Interrupt link LNKB configured for IRQ 10371gitlab # [ 0.757200] ACPI: PCI: Interrupt link LNKC configured for IRQ 11372gitlab # [ 0.759022] ACPI: PCI: Interrupt link LNKD configured for IRQ 11373gitlab # [ 0.759962] ACPI: PCI: Interrupt link LNKS configured for IRQ 9374gitlab # [ 0.763205] iommu: Default domain type: Translated375gitlab # [ 0.763535] iommu: DMA domain TLB invalidation policy: lazy mode376gitlab # [ 0.765230] ACPI: bus type USB registered377gitlab # [ 0.765966] usbcore: registered new interface driver usbfs378gitlab # [ 0.766900] usbcore: registered new interface driver hub379gitlab # [ 0.767847] usbcore: registered new device driver usb380gitlab # [ 0.769960] NetLabel: Initializing381gitlab # [ 0.770629] NetLabel: domain hash size = 128382gitlab # [ 0.771886] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO383gitlab # [ 0.772941] NetLabel: unlabeled traffic allowed by default384gitlab # [ 0.773889] PCI: Using ACPI for IRQ routing385gitlab # [ 0.775060] pci 0000:00:02.0: vgaarb: setting as boot VGA device386gitlab # [ 0.775875] pci 0000:00:02.0: vgaarb: bridge control possible387gitlab # [ 0.775875] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none388gitlab # [ 0.778884] vgaarb: loaded389gitlab # [ 0.779638] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0390gitlab # [ 0.780879] hpet0: 3 comparators, 64-bit 100.000000 MHz counter391gitlab # [ 0.787980] clocksource: Switched to clocksource kvm-clock392gitlab # [ 0.791271] VFS: Disk quotas dquot_6.6.0393gitlab # [ 0.792047] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)394gitlab # [ 0.793662] pnp: PnP ACPI init395gitlab # [ 0.794888] pnp: PnP ACPI: found 6 devices396gitlab # [ 0.803647] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns397gitlab # [ 0.805298] clocksource: Switched to clocksource acpi_pm398gitlab # [ 0.806412] NET: Registered PF_INET protocol family399gitlab # [ 0.808704] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)400gitlab # [ 0.826873] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)401gitlab # [ 0.828470] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)402gitlab # [ 0.829925] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)403gitlab # [ 0.833756] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)404gitlab # [ 0.835129] TCP: Hash tables configured (established 65536 bind 65536)405gitlab # [ 0.836390] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)406gitlab # [ 0.838119] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)407gitlab # [ 0.839631] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)408gitlab # [ 0.841014] NET: Registered PF_UNIX/PF_LOCAL protocol family409gitlab # [ 0.842017] NET: Registered PF_XDP protocol family410gitlab # [ 0.842898] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]411gitlab # [ 0.844010] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]412gitlab # [ 0.845096] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]413gitlab # [ 0.846286] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]414gitlab # [ 0.847484] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window]415gitlab # [ 0.848872] pci 0000:00:01.0: PIIX3: Enabling Passive Release416gitlab # [ 0.849890] pci 0000:00:00.0: Limiting direct PCI/PCI transfers417gitlab # [ 0.852349] ACPI: \_SB_.LNKD: Enabled at IRQ 11418gitlab # [ 0.855413] PCI: CLS 0 bytes, default 64419gitlab # [ 0.856232] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)420gitlab # [ 0.856581] Trying to unpack rootfs image as initramfs...421gitlab # [ 0.857392] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB)422gitlab # [ 0.857514] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns423gitlab # [ 0.880593] Initialise system trusted keyrings424gitlab # [ 0.881792] workingset: timestamp_bits=40 max_order=21 bucket_order=0425gitlab # [ 0.893312] Key type asymmetric registered426gitlab # [ 0.894183] Asymmetric key parser 'x509' registered427gitlab # [ 0.895129] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)428gitlab # [ 0.896766] io scheduler mq-deadline registered429gitlab # [ 0.897634] io scheduler kyber registered430gitlab # [ 0.902789] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled431gitlab # [ 0.904161] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A432gitlab # [ 0.910262] Linux agpgart interface v0.103433gitlab # [ 0.912460] ACPI: bus type drm_connector registered434gitlab # [ 0.917360] usbcore: registered new interface driver usbserial_generic435gitlab # [ 0.920149] usbserial: USB Serial support registered for generic436gitlab # [ 0.922705] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.437gitlab # [ 0.922705] Please enable it if your BIOS has the CPPC option.438gitlab # [ 0.928409] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled439gitlab # [ 0.931917] drop_monitor: Initializing network drop monitor service440gitlab # [ 0.934903] NET: Registered PF_INET6 protocol family441gitlab # [ 0.938782] Segment Routing with IPv6442gitlab # [ 0.940598] In-situ OAM (IOAM) with IPv6443gitlab # [ 0.945057] IPI shorthand broadcast: enabled444gitlab # [ 0.956555] sched_clock: Marking stable (806031148, 150369244)->(978243057, -21842665)445gitlab # [ 0.961278] registered taskstats version 1446gitlab # [ 0.964356] Loading compiled-in X.509 certificates447gitlab # [ 0.987613] Demotion targets for Node 0: null448gitlab # [ 0.989626] Key type .fscrypt registered449gitlab # [ 0.991003] Key type fscrypt-provisioning registered450gitlab # [ 0.993025] ima: No TPM chip found, activating TPM-bypass!451gitlab # [ 0.994865] ima: Allocated hash algorithm: sha1452gitlab # [ 0.996874] ima: No architecture policies found453gitlab # [ 0.998850] PM: Magic number: 2:732:931454gitlab # [ 1.001964] RAS: Correctable Errors collector initialized.455gitlab # [ 1.009701] clk: Disabling unused clocks456gitlab # [ 1.010500] PM: genpd: Disabling unused power domains457gitlab # [ 1.145097] Freeing initrd memory: 29392K458gitlab # [ 1.152705] Freeing unused decrypted memory: 2028K459gitlab # [ 1.159209] Freeing unused kernel image (initmem) memory: 3640K460gitlab # [ 1.162009] Write protecting the kernel read-only data: 32768k461gitlab # [ 1.167016] Freeing unused kernel image (text/rodata gap) memory: 1220K462gitlab # [ 1.171004] Freeing unused kernel image (rodata/data gap) memory: 760K463gitlab # [ 1.239196] x86/mm: Checked W+X mappings: passed, no W+X pages found.464gitlab # [ 1.240476] Run /init as init process465gitlab # [ 1.252539] systemd[1]: Inserted module 'autofs4'466gitlab # [ 1.275527] fuse: init (API version 7.45)467gitlab # [ 1.284104] ACPI: \_SB_.LNKC: Enabled at IRQ 10468gitlab # [ 1.290860] ACPI: \_SB_.LNKA: Enabled at IRQ 10469gitlab # [ 1.294278] ACPI: \_SB_.LNKB: Enabled at IRQ 11470gitlab # [ 1.330939] systemd[1]: Successfully made /usr/ read-only.471gitlab # [ 1.667083] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)472gitlab # [ 1.708144] systemd[1]: Detected virtualization kvm.473gitlab # [ 1.710440] systemd[1]: Detected architecture x86-64.474gitlab # [ 1.712596] systemd[1]: Running in initrd.475gitlab # [ 1.715156] systemd[1]: Initializing machine ID from random generator.476gitlab # [ 1.717761] systemd[1]: Hostname set to <gitlab>.477gitlab # [ 1.859866] systemd[1]: bpf-restrict-fs: LSM BPF program attached478gitlab # [ 1.911937] systemd[1]: Queued start job for default target Initrd Default Target.479gitlab # [ 1.923080] systemd[1]: Created slice Slice /system/modprobe.480gitlab # [ 1.924290] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.481gitlab # [ 1.925794] systemd[1]: Expecting device /dev/disk/by-label/nix-store...482gitlab # [ 1.927056] systemd[1]: Expecting device /dev/disk/by-label/nixos...483gitlab # [ 1.928248] systemd[1]: Reached target Path Units.484gitlab # [ 1.929206] systemd[1]: Reached target Slice Units.485gitlab # [ 1.930181] systemd[1]: Reached target Swaps.486gitlab # [ 1.931054] systemd[1]: Reached target Timer Units.487gitlab # [ 1.932136] systemd[1]: Listening on D-Bus System Message Bus Socket.488gitlab # [ 1.933470] systemd[1]: Listening on Journal Socket (/dev/log).489gitlab # [ 1.934712] systemd[1]: Listening on Journal Sockets.490gitlab # [ 1.935780] systemd[1]: Listening on udev Control Socket.491gitlab # [ 1.936845] systemd[1]: Listening on udev Kernel Socket.492gitlab # [ 1.937843] systemd[1]: Reached target Socket Units.493gitlab # [ 1.939907] systemd[1]: Starting Create List of Static Device Nodes...494gitlab # [ 1.942180] systemd[1]: Starting Load Kernel Module 9pnet_virtio...495gitlab # [ 1.944320] systemd[1]: Starting Load Kernel Module configfs...496gitlab # [ 1.951320] systemd[1]: Starting Journal Service...497gitlab # [ 1.955456] systemd[1]: Starting Load Kernel Modules...498gitlab # [ 1.956525] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os499gitlab # [ 1.960861] systemd[1]: Starting Coldplug All udev Devices...500gitlab # [ 1.966263] systemd[1]: Finished Create List of Static Device Nodes.501gitlab # [ 1.972148] systemd[1]: modprobe@configfs.service: Deactivated successfully.502gitlab # [ 1.976823] systemd[1]: Finished Load Kernel Module configfs.503gitlab # [ 1.979818] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config504gitlab # [ 1.982199] systemd-journald[90]: Collecting audit messages is disabled.505gitlab # [ 1.990467] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...506gitlab # [ 2.004888] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.507gitlab # [ 2.010293] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev508gitlab # [ 2.010321] netfs: FS-Cache loaded509gitlab # [ 2.018913] 9pnet: Installing 9P2000 support510gitlab # [ 2.019447] systemd[1]: Finished Load Kernel Modules.511gitlab # [ 2.023070] systemd[1]: Starting Apply Kernel Variables...512gitlab # [ 2.025743] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.513gitlab # [ 2.027568] systemd[1]: Finished Load Kernel Module 9pnet_virtio.514gitlab # [ 1.884692] systemd-modules-load[91]: Using 4 probe threads515gitlab # [ 1.886046] systemd-modules-load[91]: Inserted module 'virtio_balloon'516gitlab # [ 2.036265] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.517gitlab # [ 1.888103] systemd-modules-load[91]: Inserted module 'virtio_gpu'518gitlab # [ 2.038602] systemd[1]: Started Journal Service.519gitlab # [ 1.890096] systemd-modules-load[91]: Inserted module 'dm_mod'520gitlab # [ 1.893830] systemd[1]: Starting Create Static Device Nodes in /dev...521gitlab # [ 1.900535] systemd[1]: Finished Apply Kernel Variables.522gitlab # [ 1.906395] systemd[1]: Finished Create Static Device Nodes in /dev.523gitlab # [ 1.907428] systemd[1]: Reached target Preparation for Local File Systems.524gitlab # [ 1.908425] systemd[1]: Reached target Local File Systems.525gitlab # [ 1.909313] systemd[1]: Starting Create System Files and Directories...526gitlab # [ 1.910371] systemd[1]: Starting Rule-based Manager for Device Events and Files...527gitlab # [ 1.919734] systemd[1]: Finished Create System Files and Directories.528gitlab # [ 1.930146] systemd-udevd[111]: Using default interface naming scheme 'v261'.529gitlab # [ 1.940585] systemd[1]: Started Rule-based Manager for Device Events and Files.530gitlab # [ 1.947567] systemd[1]: Finished Coldplug All udev Devices.531gitlab # [ 1.948502] systemd[1]: Reached target System Initialization.532gitlab # [ 1.949382] systemd[1]: Reached target Basic System.533gitlab # [ 2.203701] uhci_hcd 0000:00:01.2: UHCI Host Controller534gitlab # [ 2.204542] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1535gitlab # [ 2.205671] uhci_hcd 0000:00:01.2: detected 2 ports536gitlab # [ 2.206634] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140537gitlab # [ 2.207787] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18538gitlab # [ 2.209047] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1539gitlab # [ 2.210127] usb usb1: Product: UHCI Host Controller540gitlab # [ 2.210886] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd541gitlab # [ 2.211703] usb usb1: SerialNumber: 0000:00:01.2542gitlab # [ 2.212733] hub 1-0:1.0: USB hub found543gitlab # [ 2.214254] hub 1-0:1.0: 2 ports detected544gitlab # [ 2.216733] virtio_blk virtio4: 4/0/0 default/read/poll queues545gitlab # [ 2.218803] SCSI subsystem initialized546gitlab # [ 2.219616] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)547gitlab # [ 2.251151] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12548gitlab # [ 2.254193] serio: i8042 KBD port at 0x60,0x64 irq 1549gitlab # [ 2.254950] serio: i8042 AUX port at 0x60,0x64 irq 12550gitlab # [ 2.256074] virtio_blk virtio5: 4/0/0 default/read/poll queues551gitlab # [ 2.258553] scsi host0: ata_piix552gitlab # [ 2.260613] scsi host1: ata_piix553gitlab # [ 2.261324] virtio_blk virtio5: [vdb] 11953416 512-byte logical blocks (6.12 GB/5.70 GiB)554gitlab # [ 2.261333] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0555gitlab # [ 2.267283] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0556gitlab # [ 2.139634] systemd[1]: Starting Virtual Console Setup...557gitlab # [ 2.145277] (udev-worker)[139]: Network interface NamePolicy= disabled on kernel command line.558gitlab # [ 2.148131] (udev-worker)[135]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.559gitlab # [ 2.150419] (udev-worker)[135]: Network interface NamePolicy= disabled on kernel command line.560gitlab # [ 2.163859] systemd-vconsole-setup[151]: Configuration of first virtual console was skipped, ignoring remaining ones.561gitlab # [ 2.165639] systemd[1]: Finished Virtual Console Setup.562gitlab # [ 2.425739] ata2: found unknown device (class 0)563gitlab # [ 2.428724] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100564gitlab # [ 2.432993] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5565gitlab # [ 2.436536] usb 1-1: new full-speed USB device number 2 using uhci_hcd566gitlab # [ 2.464733] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0567gitlab # [ 2.339207] systemd[1]: Found device /dev/disk/by-label/nixos.568gitlab # [ 2.345798] systemd[1]: Found device /dev/disk/by-label/nix-store.569gitlab # [ 2.348222] systemd[1]: Reached target Initrd Root Device.570gitlab # [ 2.350447] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...571gitlab # [ 2.505727] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray572gitlab # [ 2.393340] systemd-fsck[160]: nixos: clean, 12/65536 files, 13019/262144 blocks573gitlab # [ 2.546955] cdrom: Uniform CD-ROM driver Revision: 3.20574gitlab # [ 2.401679] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.575gitlab # [ 2.605404] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00576gitlab # [ 2.608504] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10577gitlab # [ 2.611625] usb 1-1: Product: QEMU USB Tablet578gitlab # [ 2.613423] usb 1-1: Manufacturer: QEMU579gitlab # [ 2.614938] usb 1-1: SerialNumber: 28754-0000:00:01.2-1580gitlab # [ 2.647885] hid: raw HID events driver (C) Jiri Kosina581gitlab # [ 2.662332] usbcore: registered new interface driver usbhid582gitlab # [ 2.664471] usbhid: USB HID core driver583gitlab # [ 2.673661] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2584gitlab # [ 2.677247] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0585gitlab # [ 2.815554] systemd[1]: Mounting /sysroot...586gitlab # [ 3.119204] EXT4-fs (vda): mounted filesystem 7d9b8c44-852b-4536-ae0c-a9b16979dd1c r/w with ordered data mode. Quota mode: none.587gitlab # [ 2.973874] systemd[1]: Mounted /sysroot.588gitlab # [ 2.975064] systemd[1]: Reached target Initrd Root File System.589gitlab # [ 2.978819] systemd[1]: Mounting /sysroot/nix/.ro-store...590gitlab # [ 2.980897] systemd[1]: Mounting /sysroot/nix/.rw-store...591gitlab # [ 2.990775] systemd[1]: Mounting /sysroot/run...592gitlab # [ 2.993168] systemd[1]: Mounting /sysroot/tmp/shared...593gitlab # [ 3.000627] systemd[1]: Mounting /sysroot/tmp/xchg...594gitlab # [ 3.007254] systemd[1]: Starting Mountpoints Configured in the Real Root...595gitlab # [ 3.158837] 9p: Installing v9fs 9p2000 file system support596gitlab # [ 3.011098] systemd[1]: Mounted /sysroot/nix/.rw-store.597gitlab # [ 3.013076] systemd[1]: Mounted /sysroot/run.598gitlab # [ 3.014924] systemd[1]: Mounted /sysroot/tmp/shared.599gitlab # [ 3.018285] systemd[1]: Starting rw-sysroot-nix-store.service...600gitlab # [ 3.170897] erofs (device vdb): mounted with root inode @ nid 128.601gitlab # [ 3.023041] systemd[1]: Mounted /sysroot/nix/.ro-store.602gitlab # [ 3.024941] systemd[1]: Mounted /sysroot/tmp/xchg.603gitlab # [ 3.028049] systemd-sysroot-fstab-check[181]: /sysroot should be mounted in the initrd, will request daemon-reload.604gitlab # [ 3.029752] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.605gitlab # [ 3.031105] systemd[1]: Finished rw-sysroot-nix-store.service.606gitlab # [ 3.033599] systemd[1]: Reload requested from client PID 181 ('systemd-sysroot') (unit initrd-parse-etc.service)...607gitlab # [ 3.035335] systemd[1]: Reloading...608gitlab # [ 3.099683] systemd[1]: Reloading finished in 64 ms.609gitlab # [ 3.120673] systemd-sysroot-fstab-check[181]: Requesting initrd-fs.target/start/replace...610gitlab # [ 3.123464] systemd-sysroot-fstab-check[181]: Requesting swap.target/start/replace...611gitlab # [ 3.125992] systemd[1]: Starting rw-sysroot-nix-store.service...612gitlab # [ 3.128340] systemd[1]: initrd-parse-etc.service: Deactivated successfully.613gitlab # [ 3.130665] systemd[1]: Finished Mountpoints Configured in the Real Root.614gitlab # [ 3.132876] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.615gitlab # [ 3.165691] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.616gitlab # [ 3.168198] systemd[1]: Finished rw-sysroot-nix-store.service.617gitlab # [ 3.816191] systemd[1]: Mounting /sysroot/nix/store...618gitlab # [ 3.854552] systemd[1]: Mounted /sysroot/nix/store.619gitlab # [ 3.855821] systemd[1]: Reached target Initrd File Systems.620gitlab # [ 3.857887] systemd[1]: Starting Find NixOS closure...621gitlab # [ 3.859679] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...622gitlab # [ 3.885135] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.623gitlab # [ 3.887616] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.624gitlab # [ 3.896130] systemd[1]: Finished Find NixOS closure.625gitlab # [ 3.897185] systemd[1]: Reached target Initrd Default Target.626gitlab # [ 3.899121] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...627gitlab # [ 3.929920] systemd[1]: Stopped target Initrd Default Target.628gitlab # [ 3.931103] systemd[1]: Stopped target Basic System.629gitlab # [ 3.932175] systemd[1]: Stopped target Initrd Root Device.630gitlab # [ 3.933234] systemd[1]: Stopped target Path Units.631gitlab # [ 3.934248] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.632gitlab # [ 3.935511] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.633gitlab # [ 3.936884] systemd[1]: Stopped target Slice Units.634gitlab # [ 3.937819] systemd[1]: Stopped target Socket Units.635gitlab # [ 3.938688] systemd[1]: Stopped target System Initialization.636gitlab # [ 3.939697] systemd[1]: Stopped target Swaps.637gitlab # [ 3.940576] systemd[1]: Stopped target Timer Units.638gitlab # [ 3.941503] systemd[1]: dbus.socket: Deactivated successfully.639gitlab # [ 3.942540] systemd[1]: Closed D-Bus System Message Bus Socket.640gitlab # [ 3.943637] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.641gitlab # [ 3.944839] systemd[1]: Stopped Find NixOS closure.642gitlab # [ 3.945688] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio643gitlab # [ 3.947369] systemd[1]: Starting rw-sysroot-nix-store.service...644gitlab # [ 3.948438] systemd[1]: systemd-sysctl.service: Deactivated successfully.645gitlab # [ 3.950350] systemd[1]: Stopped Apply Kernel Variables.646gitlab # [ 3.951326] systemd[1]: systemd-modules-load.service: Deactivated successfully.647gitlab # [ 3.953218] systemd[1]: Stopped Load Kernel Modules.648gitlab # [ 3.954262] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.649gitlab # [ 3.956318] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.650gitlab # [ 3.957654] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.651gitlab # [ 3.959493] systemd[1]: Stopped Create System Files and Directories.652gitlab # [ 3.960648] systemd[1]: Stopped target Local File Systems.653gitlab # [ 3.961636] systemd[1]: Stopped target Preparation for Local File Systems.654gitlab # [ 3.962793] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.655gitlab # [ 3.964072] systemd[1]: Stopped Coldplug All udev Devices.656gitlab # [ 3.965044] systemd[1]: Stopping Rule-based Manager for Device Events and Files...657gitlab # [ 3.966407] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.658gitlab # [ 3.967686] systemd[1]: Stopped Virtual Console Setup.659gitlab # [ 3.968588] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.660gitlab # [ 3.969934] systemd[1]: Finished rw-sysroot-nix-store.service.661gitlab # [ 3.971234] systemd[1]: systemd-udevd.service: Deactivated successfully.662gitlab # [ 3.972319] systemd[1]: Stopped Rule-based Manager for Device Events and Files.663gitlab # [ 3.973584] systemd[1]: initrd-cleanup.service: Deactivated successfully.664gitlab # [ 3.975312] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.665gitlab # [ 3.978157] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.666gitlab # [ 3.979369] systemd[1]: Closed udev Control Socket.667gitlab # [ 3.980630] systemd[1]: Starting Cleanup udev Database...668gitlab # [ 3.981743] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.669gitlab # [ 3.983116] systemd[1]: Stopped Create Static Device Nodes in /dev.670gitlab # [ 3.984210] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.671gitlab # [ 3.985579] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.672gitlab # [ 3.986845] systemd[1]: kmod-static-nodes.service: Deactivated successfully.673gitlab # [ 3.988106] systemd[1]: Stopped Create List of Static Device Nodes.674gitlab # [ 4.008414] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.675gitlab # [ 4.009745] systemd[1]: Finished Cleanup udev Database.676gitlab # [ 4.010665] systemd[1]: Reached target Switch Root.677gitlab # [ 4.012162] systemd[1]: Starting NixOS Activation...678gitlab # [ 4.117127] initrd-nixos-activation-start[226]: booting system configuration /nix/store/2w0b254hl61bswmdix102sqs162p3ip5-nixos-system-gitlab-test679gitlab # [ 4.149567] initrd-nixos-activation-start[226]: running activation script...680gitlab # [ 4.333067] initrd-nixos-activation-start[249]: setting up /etc...681gitlab # [ 4.444678] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.682gitlab # [ 4.446048] systemd[1]: Finished NixOS Activation.683gitlab # [ 4.448968] systemd[1]: Starting Switch Root...684gitlab # [ 4.481757] systemd[1]: Switching root.685gitlab # [ 4.691328] systemd-journald[90]: Received SIGTERM from PID 1 (systemd).686gitlab # [ 4.764399] NET: Registered PF_VSOCK protocol family687gitlab # [ 5.128533] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)688gitlab # [ 5.136631] systemd[1]: Detected virtualization kvm.689gitlab # [ 5.138107] systemd[1]: Detected architecture x86-64.690gitlab # [ 5.139673] systemd[1]: Detected first boot.691gitlab # [ 5.141712] systemd[1]: Initializing machine ID from random generator.692gitlab # [ 5.277969] systemd[1]: bpf-restrict-fs: LSM BPF program attached693gitlab # [ 5.358614] systemd[1]: Applying preset policy.694gitlab # [ 5.527797] systemd[1]: Populated /etc with preset unit settings.695gitlab # [ 5.640741] systemd[1]: initrd-switch-root.service: Deactivated successfully.696gitlab # [ 5.642305] systemd[1]: Stopped initrd-switch-root.service.697gitlab # [ 5.644781] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.698gitlab # [ 5.646983] systemd[1]: Created slice Slice /system/getty.699gitlab # [ 5.648412] systemd[1]: Created slice GitLab DevOps Platform Slice.700gitlab # [ 5.649984] systemd[1]: Created slice User and Session Slice.701gitlab # [ 5.650927] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.702gitlab # [ 5.652185] systemd[1]: Started Forward Password Requests to Wall Directory Watch.703gitlab # [ 5.653509] systemd[1]: Expecting device /dev/hvc0...704gitlab # [ 5.654336] systemd[1]: Expecting device /dev/ttyS0...705gitlab # [ 5.655231] systemd[1]: Reached target Local Encrypted Volumes.706gitlab # [ 5.656106] systemd[1]: Stopped target initrd-fs.target.707gitlab # [ 5.656885] systemd[1]: Stopped target initrd-root-fs.target.708gitlab # [ 5.657773] systemd[1]: Stopped target initrd-switch-root.target.709gitlab # [ 5.658690] systemd[1]: Reached target Virtual Machines and Containers.710gitlab # [ 5.659670] systemd[1]: Reached target Path Units.711gitlab # [ 5.660412] systemd[1]: Reached target Remote File Systems.712gitlab # [ 5.661237] systemd[1]: Reached target Slice Units.713gitlab # [ 5.662023] systemd[1]: Reached target Swaps.714gitlab # [ 5.663456] systemd[1]: Listening on Query the User Interactively for a Password.715gitlab # [ 5.665435] systemd[1]: Listening on Process Core Dump Socket.716gitlab # [ 5.666806] systemd[1]: Listening on Credential Encryption/Decryption.717gitlab # [ 5.668337] systemd[1]: Listening on Factory Reset Management.718gitlab # [ 5.669298] systemd[1]: Listening on Hostname Service Socket.719gitlab # [ 5.671658] systemd[1]: Starting Journal Log Access Socket...720gitlab # [ 5.673034] systemd[1]: Listening on Journal Audit Socket.721gitlab # [ 5.675005] systemd[1]: Listening on Console Output Muting Service Socket.722gitlab # [ 5.676208] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.723gitlab # [ 5.677412] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os724gitlab # [ 5.678867] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki725gitlab # [ 5.681674] systemd[1]: Listening on Disk Repartitioning Service Socket.726gitlab # [ 5.682737] systemd[1]: Listening on udev Control Socket.727gitlab # [ 5.683639] systemd[1]: Listening on udev Varlink Socket.728gitlab # [ 5.686296] systemd[1]: Mounting Huge Pages File System...729gitlab # [ 5.688612] systemd[1]: Mounting POSIX Message Queue File System...730gitlab # [ 5.698988] systemd[1]: Mounting Kernel Debug File System...731gitlab # [ 5.702106] systemd[1]: Mounting Kernel Trace File System...732gitlab # [ 5.704646] systemd[1]: Starting Create List of Static Device Nodes...733gitlab # [ 5.706879] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio734gitlab # [ 5.718166] systemd[1]: Starting Load Kernel Module configfs...735gitlab # [ 5.720641] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm736gitlab # [ 5.724464] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore737gitlab # [ 5.728498] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse738gitlab # [ 5.734458] systemd[1]: Mounting FUSE Control File System...739gitlab # [ 5.763277] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67740gitlab # [ 5.775501] systemd[1]: Starting Journal Service...741gitlab # [ 5.779066] systemd[1]: Starting Load Kernel Modules...742gitlab # [ 5.788301] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...743gitlab # [ 5.792559] systemd[1]: Starting Remount Root and Kernel File Systems...744gitlab # [ 5.795514] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os745gitlab # [ 5.803640] systemd-journald[320]: Collecting audit messages is enabled.746gitlab # [ 5.805434] systemd[1]: Starting Coldplug All udev Devices...747gitlab # [ 5.808484] systemd[1]: Listening on Journal Log Access Socket.748gitlab # [ 5.809570] systemd[1]: Mounted Huge Pages File System.749gitlab # [ 5.810403] loop: module loaded750gitlab # [ 5.810506] EXT4-fs (vda): re-mounted 7d9b8c44-852b-4536-ae0c-a9b16979dd1c.751gitlab # [ 5.810532] systemd[1]: Mounted POSIX Message Queue File System.752gitlab # [ 5.810690] systemd[1]: Mounted Kernel Debug File System.753gitlab # [ 5.815222] systemd[1]: Mounted Kernel Trace File System.754gitlab # [ 5.816709] systemd[1]: Finished Create List of Static Device Nodes.755gitlab # [ 5.818011] systemd[1]: modprobe@configfs.service: Deactivated successfully.756gitlab # [ 5.819582] systemd[1]: Finished Load Kernel Module configfs.757gitlab # [ 5.669973] systemd[1]: Queued start job for default target Multi-User System.758gitlab # [ 5.671340] systemd[1]: Unnecessary job was removed for /dev/vdb.759gitlab # [ 5.821640] systemd[1]: Started Journal Service.760gitlab # [ 5.672693] systemd[1]: systemd-journald.service: Deactivated successfully.761gitlab # [ 5.674387] systemd-modules-load[321]: Using 3 probe threads762gitlab # [ 5.675584] systemd-modules-load[321]: Inserted module 'tls'763gitlab # [ 5.676652] systemd-modules-load[321]: Inserted module 'loop'764gitlab # [ 5.677720] systemd[1]: Mounted FUSE Control File System.765gitlab # [ 5.678694] systemd[1]: Finished Load Kernel Modules.766gitlab # [ 5.679583] systemd[1]: Finished Remount Root and Kernel File Systems.767gitlab # [ 5.682785] systemd[1]: Listening on Disk Image Download Service Socket.768gitlab # [ 5.684210] systemd[1]: Mounting Kernel Configuration File System...769gitlab # [ 5.688766] systemd[1]: Starting Firewall...770gitlab # [ 5.691120] systemd-oomd[323]: No swap; memory pressure usage will be degraded771gitlab # [ 5.696209] systemd[1]: Starting Flush Journal to Persistent Storage...772gitlab # [ 5.697247] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore773gitlab # [ 5.699076] systemd[1]: Starting Load/Save OS Random Seed...774gitlab # [ 5.700097] systemd[1]: Starting Apply Kernel Variables...775gitlab # [ 5.700982] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...776gitlab # [ 5.702670] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os777gitlab # [ 5.704034] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.778gitlab # [ 5.855502] systemd-journald[320]: Received client request to flush runtime journal.779gitlab # [ 5.750895] systemd[1]: Mounted Kernel Configuration File System.780gitlab # [ 5.752058] systemd[1]: Finished Apply Kernel Variables.781gitlab # [ 5.752973] systemd[1]: Finished Load/Save OS Random Seed.782gitlab # [ 5.754059] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.783gitlab # [ 5.755272] systemd[1]: Reached target First Boot Complete.784gitlab # [ 5.756252] systemd[1]: Starting Create Static Device Nodes in /dev...785gitlab # [ 5.757340] systemd[1]: Finished Flush Journal to Persistent Storage.786gitlab # [ 5.758284] systemd[1]: Finished Create Static Device Nodes in /dev.787gitlab # [ 5.759219] systemd[1]: Reached target Preparation for Local File Systems.788gitlab # [ 5.760190] systemd[1]: Starting Rule-based Manager for Device Events and Files...789gitlab # [ 5.785440] systemd-udevd[362]: Using default interface naming scheme 'v261'.790gitlab # [ 5.814106] systemd[1]: Started Rule-based Manager for Device Events and Files.791gitlab # [ 5.818503] systemd[1]: Finished Coldplug All udev Devices.792gitlab # [ 5.879387] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse793gitlab # [ 5.911630] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.794gitlab # [ 5.928295] (udev-worker)[427]: Network interface NamePolicy= disabled on kernel command line.795gitlab # [ 5.942743] systemd[1]: Finished Firewall.796gitlab # [ 5.943763] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped.797gitlab # [ 5.951662] (udev-worker)[423]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.798gitlab # [ 5.953854] (udev-worker)[423]: Network interface NamePolicy= disabled on kernel command line.799gitlab # [ 5.964825] systemd[1]: Condition check resulted in Virtio network device being skipped.800gitlab # [ 5.966832] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore801gitlab # [ 5.968353] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67802gitlab # [ 5.970540] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore803gitlab # [ 5.972174] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os804gitlab # [ 5.973461] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os805gitlab # [ 6.135767] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0806gitlab # [ 6.144460] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD807gitlab # [ 6.147027] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3808gitlab # [ 6.149811] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console809gitlab # [ 6.154481] ACPI: button: Power Button [PWRF]810gitlab # [ 6.163097] Console: switching to colour dummy device 80x25811gitlab # [ 6.164328] [drm] Found bochs VGA, ID 0xb0c5.812gitlab # [ 6.164330] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000.813gitlab # [ 6.166152] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic814gitlab # [ 6.167160] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0815gitlab # [ 6.174120] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4816gitlab # [ 6.188853] Console: switching to colour frame buffer device 160x50817gitlab # [ 6.190287] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device818gitlab # [ 6.195577] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6819gitlab # [ 6.198235] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5820gitlab # [ 6.213970] rtc_cmos 00:05: RTC can wake from S4821gitlab # [ 6.215719] Floppy drive(s): fd0 is 2.88M AMI BIOS822gitlab # [ 6.222805] rtc_cmos 00:05: registered as rtc0823gitlab # [ 6.223508] rtc_cmos 00:05: setting system clock to 2026-11-10T10:54:31 UTC (1794308071)824gitlab # [ 6.224533] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs825gitlab # [ 6.225034] parport_pc 00:03: reported by Plug and Play ACPI826gitlab # [ 6.226772] FDC 0 is a S82078B827gitlab # [ 6.231551] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)]828gitlab # [ 6.233270] systemd-journald[320]: Time jumped backwards, rotating.829gitlab # [ 6.236201] mousedev: PS/2 mouse device common for all mice830gitlab # [ 6.161563] systemd[1]: Starting Virtual Console Setup...831gitlab # [ 6.180270] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.832gitlab # [ 6.184176] systemd[1]: Stopped Virtual Console Setup.833gitlab # [ 6.197810] systemd[1]: Starting Virtual Console Setup...834gitlab # [ 6.201559] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.835gitlab # [ 6.202761] systemd[1]: Stopped Virtual Console Setup.836gitlab # [ 6.206176] systemd[1]: Starting Virtual Console Setup...837gitlab # [ 6.357713] ppdev: user-space parallel port driver838gitlab # [ 6.377498] kvm_amd: TSC scaling supported839gitlab # [ 6.378006] kvm_amd: Nested Virtualization enabled840gitlab # [ 6.378691] kvm_amd: Nested Paging enabled841gitlab # [ 6.379192] kvm_amd: LBR virtualization supported842gitlab # [ 6.379847] kvm_amd: Virtual VMLOAD VMSAVE supported843gitlab # [ 6.380555] kvm_amd: Virtual GIF supported844gitlab # [ 6.381045] kvm_amd: Virtual NMI enabled845gitlab # [ 6.413021] EDAC MC: Ver: 3.0.0846gitlab # [ 6.267890] systemd-vconsole-setup[493]: Configuration of first virtual console was skipped, ignoring remaining ones.847gitlab # [ 6.270447] systemd[1]: Finished Virtual Console Setup.848gitlab # [ 6.495416] systemd[1]: Mounting /run/wrappers...849gitlab # [ 6.519099] systemd[1]: Mounted /run/wrappers.850gitlab # [ 6.521001] systemd[1]: Reached target Local File Systems.851gitlab # [ 6.523136] systemd[1]: Listening on Boot Loader Control Service Socket.852gitlab # [ 6.525561] systemd[1]: Starting register-nix-paths.service...853gitlab # [ 6.527734] systemd[1]: Starting Create SUID/SGID Wrappers...854gitlab # [ 6.529913] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.855gitlab # [ 6.533099] systemd[1]: Starting Save Transient machine-id to Disk...856gitlab # [ 6.539211] systemd[1]: Starting Create System Files and Directories...857gitlab # [ 6.576926] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.858gitlab # [ 6.579861] systemd[1]: Finished Create System Files and Directories.859gitlab # [ 6.585846] systemd[1]: Finished Save Transient machine-id to Disk.860gitlab # [ 6.595214] systemd[1]: Starting Rebuild Journal Catalog...861gitlab # [ 6.597482] systemd[1]: Starting Record System Boot/Shutdown in UTMP...862gitlab # [ 6.612807] systemd[1]: Finished Record System Boot/Shutdown in UTMP.863gitlab # [ 6.632703] systemd[1]: Finished Rebuild Journal Catalog.864gitlab # [ 6.634802] systemd[1]: Starting Update is Completed...865gitlab # [ 6.652492] systemd[1]: Finished Update is Completed.866gitlab # [ 6.691939] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.867gitlab # [ 6.694645] systemd[1]: Finished Create SUID/SGID Wrappers.868gitlab # [ 6.923097] systemd[1]: Finished register-nix-paths.service.869gitlab # [ 6.925327] systemd[1]: Reached target System Initialization.870gitlab # [ 6.927405] systemd[1]: Started Discard unused filesystem blocks once a week.871gitlab # [ 6.929847] systemd[1]: Started logrotate.timer.872gitlab # [ 6.931505] systemd[1]: Started Daily Cleanup of Temporary Directories.873gitlab # [ 6.933709] systemd[1]: Reached target Timer Units.874gitlab # [ 6.935470] systemd[1]: Listening on D-Bus System Message Bus Socket.875gitlab # [ 6.937637] systemd[1]: Listening on Nix Daemon Socket.876gitlab # [ 6.939488] systemd[1]: Starting nixbot CI service socket...877gitlab # [ 6.941481] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.878gitlab # [ 6.944694] systemd[1]: Starting D-Bus System Message Bus...879gitlab # [ 6.946862] systemd[1]: Listening on nixbot CI service socket.880gitlab # [ 6.948763] systemd[1]: Reached target Socket Units.881gitlab # [ 6.990466] dbus-broker-launch[575]: Looking up NSS user entry for 'systemd-timesync'...882gitlab # [ 6.993761] dbus-broker-launch[575]: NSS returned no entry for 'systemd-timesync'883gitlab # [ 6.996589] dbus-broker-launch[575]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"884gitlab # [ 7.001717] systemd[1]: Started D-Bus System Message Bus.885gitlab # [ 7.018744] dbus-broker-launch[575]: Ready886gitlab # [ 7.021195] systemd[1]: Reached target Basic System.887gitlab # [ 7.023637] systemd[1]: Started backdoor.service.888gitlab # [ 7.028131] systemd[1]: Starting gitlab-config.service...889gitlab # [ 7.035195] systemd[1]: Starting Import lastlog data into lastlog2 database...890gitlab # [ 7.038514] systemd[1]: Starting Logrotate configuration check...891gitlab # [ 7.041834] systemd[1]: Starting Name Service Cache Daemon (nsncd)...892gitlab # [ 7.044600] systemd[1]: Starting Post-Boot Actions...893gitlab # [ 7.050718] systemd[1]: Started Reset console on configuration changes.894gitlab # [ 7.054645] systemd[1]: Starting resolvconf update...895gitlab # connecting to host...896gitlab # [ 7.065942] logrotate[580]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.897gitlab # [ 7.066641] logrotate[580]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf898gitlab # [ 7.067621] logrotate[580]: note: 'monthly' overrides previously specified 'weekly'899gitlab # [ 7.069200] systemd[1]: Finished Post-Boot Actions.900gitlab: Guest shell says: b'Spawning backdoor root shell...\n'901gitlab # [ 7.076243] logrotate[580]: Reading state from file: /var/lib/logrotate.status902gitlab # [ 7.076661] logrotate[580]: state file /var/lib/logrotate.status does not exist903gitlab: connected to guest root shell904gitlab # [ 7.077656] logrotate[580]: Allocating hash table for state file, size 64 entries905gitlab: (connecting took 7.71 seconds)906gitlab # [ 7.078281] logrotate[580]: Handling 4 logs907gitlab: (finished: waiting for the VM to finish booting, in 7.71 seconds)908gitlab # [ 7.079213] logrotate[580]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed909gitlab # [ 7.079738] logrotate[580]: considering log /var/log/btmp910gitlab # [ 7.080052] logrotate[580]: Creating new state911gitlab # [ 7.080646] logrotate[580]: Now: 2026-11-10 10:54912gitlab # [ 7.081511] logrotate[580]: Last rotated at 2026-11-10 10:00913gitlab # [ 7.082407] logrotate[580]: log does not need rotating (log has already been rotated)914gitlab # [ 7.083026] logrotate[580]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed915gitlab # [ 7.083601] logrotate[580]: considering log /var/log/wtmp916gitlab # [ 7.084456] logrotate[580]: Creating new state917gitlab # [ 7.085337] logrotate[580]: Now: 2026-11-10 10:54918gitlab # [ 7.086219] logrotate[580]: Last rotated at 2026-11-10 10:00919gitlab # [ 7.087106] logrotate[580]: log does not need rotating (log has already been rotated)920gitlab # [ 7.087723] logrotate[580]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed921gitlab # [ 7.088417] logrotate[580]: switching euid from 0 to 165 and egid from 0 to 165 (pid 580)922gitlab # [ 7.088872] logrotate[580]: considering log /var/gitlab/state/log/*.log923gitlab # [ 7.089447] logrotate[580]: log /var/gitlab/state/log/*.log does not exist -- skipping924gitlab # [ 7.090033] logrotate[580]: Creating new state925gitlab # [ 7.090533] logrotate[580]: switching euid from 165 to 0 and egid from 165 to 0 (pid 580)926gitlab # [ 7.091087] logrotate[580]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed927gitlab # [ 7.091400] logrotate[580]: switching euid from 0 to 60 and egid from 0 to 60 (pid 580)928gitlab # [ 7.091887] logrotate[580]: considering log /var/log/nginx/*.log929gitlab # [ 7.092567] logrotate[580]: log /var/log/nginx/*.log does not exist -- skipping930gitlab # [ 7.093131] logrotate[580]: Creating new state931gitlab # [ 7.093630] logrotate[580]: not running postrotate script, since no logs were rotated932gitlab # [ 7.094097] logrotate[580]: switching euid from 60 to 0 and egid from 60 to 0 (pid 580)933gitlab # [ 7.117948] systemd[1]: Finished Logrotate configuration check.934gitlab # [ 7.127223] systemd[1]: Finished Import lastlog data into lastlog2 database.935gitlab # [ 7.135300] systemd[1]: Started Name Service Cache Daemon (nsncd).936gitlab # [ 7.135792] systemd[1]: Reached target Host and Network Name Lookups.937gitlab # [ 7.137026] systemd[1]: Reached target User and Group Name Lookups.938gitlab # [ 7.137994] nsncd[581]: Nov 10 10:54:32.564 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"939gitlab # [ 7.139944] systemd[1]: Starting User Login Management...940gitlab # [ 7.166561] systemd[1]: Stopped target Host and Network Name Lookups.941gitlab # [ 7.167395] systemd[1]: Stopping Host and Network Name Lookups...942gitlab # [ 7.167835] systemd[1]: Stopped target User and Group Name Lookups.943gitlab # [ 7.168497] systemd[1]: Stopping User and Group Name Lookups...944gitlab # [ 7.169092] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...945gitlab # [ 7.171774] systemd[1]: nscd.service: Deactivated successfully.946gitlab # [ 7.171972] systemd[1]: Stopped Name Service Cache Daemon (nsncd).947gitlab # [ 7.172626] systemd[1]: nscd.service: Consumed 17ms CPU time over 129ms wall clock time, 7.2M memory peak, 5.5M read from disk.948gitlab # [ 7.177451] systemd[1]: Starting Name Service Cache Daemon (nsncd)...949gitlab # [ 7.182348] systemd-logind[634]: New seat seat0.950gitlab # [ 7.183538] systemd-logind[634]: Watching system buttons on /dev/input/event2 (Power Button)951gitlab # [ 7.184870] systemd-logind[634]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)952gitlab # [ 7.186479] systemd-logind[634]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)953gitlab # [ 7.188118] systemd[1]: Started User Login Management.954gitlab # [ 7.189077] systemd[1]: Starting linger-users.service...955gitlab # [ 7.199763] systemd[1]: Started Name Service Cache Daemon (nsncd).956gitlab # [ 7.200757] systemd[1]: Reached target Host and Network Name Lookups.957gitlab # [ 7.201449] systemd[1]: Reached target User and Group Name Lookups.958gitlab # [ 7.202846] nsncd[669]: Nov 10 10:54:32.627 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"959gitlab # [ 7.207470] systemd[1]: linger-users.service: Deactivated successfully.960gitlab # [ 7.207817] systemd[1]: Finished linger-users.service.961gitlab # [ 7.211535] systemd[1]: Finished resolvconf update.962gitlab # [ 7.212219] systemd[1]: Reached target Preparation for Network.963gitlab # [ 7.214371] systemd[1]: Starting DHCP Client...964gitlab # [ 7.218353] systemd[1]: Starting Address configuration of eth1...965gitlab # [ 7.220780] systemd[1]: Starting Extra networking commands....966gitlab # [ 7.245197] network-addresses-eth1-start[698]: adding address 192.168.1.1/24... done967gitlab # [ 7.247859] network-addresses-eth1-start[698]: adding address 2001:db8:1::1/64... done968gitlab # [ 7.255434] systemd[1]: Finished Address configuration of eth1.969gitlab # [ 7.283874] systemd[1]: Finished Extra networking commands..970gitlab # [ 7.285238] systemd[1]: Reached target Network.971gitlab # [ 7.286511] systemd[1]: Starting gitlab-workhorse.service...972gitlab # [ 7.288357] systemd[1]: Starting Nginx Web Server...973gitlab # [ 7.289156] dhcpcd[707]: dhcpcd-10.3.2 starting974gitlab # [ 7.290845] dhcpcd[765]: dev: loaded udev975gitlab # [ 7.447006] 8021q: 802.1Q VLAN Support v1.8976gitlab # [ 7.297207] systemd[1]: Start[ 7.447792] 8021q: adding VLAN 0 to HW filter on device eth1977gitlab # ing PostgreSQL Server...978gitlab # [ 7.299085] systemd[1]: Starting Redis Server - redis-gitlab...979gitlab # [ 7.300648] systemd[1]: Starting Permit User Sessions...980gitlab # [ 7.307520] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.981gitlab # [ 7.319756] systemd[1]: Finished Permit User Sessions.982gitlab # [ 7.327289] systemd[1]: Started Getty on tty1.983gitlab # [ 7.328286] systemd[1]: Reached target Login Prompts.984gitlab # [ 7.527684] cfg80211: Loading compiled-in X.509 certificates for regulatory database985gitlab # [ 7.534509] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'986gitlab # [ 7.535560] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'987gitlab # [ 7.537299] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2988gitlab # [ 7.540375] cfg80211: failed to load regulatory.db989gitlab # [ 7.573552] 8021q: adding VLAN 0 to HW filter on device eth0990gitlab # [ 7.424754] dhcpcd[765]: eth0: waiting for carrier991gitlab # [ 7.425745] dhcpcd[765]: eth0: carrier acquired992gitlab # [ 7.428584] dhcpcd[765]: DUID 00:01:00:01:32:85:b4:68:52:54:00:12:34:56993gitlab # [ 7.428975] dhcpcd[765]: eth0: IAID 00:12:34:56994gitlab # [ 7.429843] redis-server[786]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.995gitlab # [ 7.430594] redis-server[786]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo996gitlab # [ 7.431156] redis-server[786]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=786, just started997gitlab # [ 7.431276] redis-server[786]: Configuration loaded998gitlab # [ 7.431898] redis[786]: monotonic clock: POSIX clock_gettime999gitlab # [ 7.432796] dhcpcd[765]: eth0: adding address fe80::5054:ff:fe12:34561000gitlab # [ 7.433548] redis[786]: Running mode=standalone, port=0.1001gitlab # [ 7.434211] systemd[1]: Started Redis Server - redis-gitlab.1002gitlab # [ 7.434832] redis[786]: Server initialized1003gitlab # [ 7.434983] redis[786]: Ready to accept connections unix1004gitlab # [ 7.435584] redis[786]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.1005gitlab # [ 7.438179] nginx-pre-start[793]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok1006gitlab # [ 7.438902] nginx-pre-start[793]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful1007gitlab # [ 7.441299] postgresql-pre-start[789]: The files belonging to this database system will be owned by user "postgres".1008gitlab # [ 7.442222] postgresql-pre-start[789]: This user must also own the server process.1009gitlab # [ 7.445730] postgresql-pre-start[789]: The database cluster will be initialized with locale "en_US.UTF-8".1010gitlab # [ 7.447491] postgresql-pre-start[789]: The default database encoding has accordingly been set to "UTF8".1011gitlab # [ 7.448418] postgresql-pre-start[789]: The default text search configuration will be set to "english".1012gitlab # [ 7.448988] postgresql-pre-start[789]: Data page checksums are enabled.1013gitlab # [ 7.450533] postgresql-pre-start[789]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok1014gitlab # [ 7.451065] postgresql-pre-start[789]: creating subdirectories ... ok1015gitlab # [ 7.451546] postgresql-pre-start[789]: selecting dynamic shared memory implementation ... posix1016gitlab # [ 7.458088] systemd[1]: Started Nginx Web Server.1017gitlab # [ 7.479463] postgresql-pre-start[789]: selecting default "max_connections" ... 1001018gitlab # [ 7.513835] postgresql-pre-start[789]: selecting default "shared_buffers" ... 128MB1019gitlab # [ 7.770701] systemd[1]: Started gitlab-workhorse.service.1020gitlab # [ 7.897854] postgresql-pre-start[789]: selecting default time zone ... UTC1021gitlab # [ 7.899663] postgresql-pre-start[789]: creating configuration files ... ok1022gitlab # [ 7.975333] gitlab-workhorse[823]: time="2026-11-10T10:54:33Z" level=info msg=Starting build_time=19700101.000000 version=19.0.41023gitlab # [ 7.976094] gitlab-workhorse[823]: time="2026-11-10T10:54:33Z" level=info msg="Using redis/go-redis"1024gitlab # [ 7.977645] gitlab-workhorse[823]: time="2026-11-10T10:54:33Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix1025gitlab # [ 8.066569] postgresql-pre-start[789]: running bootstrap script ... ok1026gitlab # [ 8.318246] dhcpcd[765]: eth0: soliciting a DHCP lease1027gitlab # [ 8.474497] NET: Registered PF_PACKET protocol family1028gitlab # [ 8.331725] dhcpcd[765]: eth0: offered 10.0.2.15 from 10.0.2.21029gitlab # [ 8.337248] dhcpcd[765]: eth0: probing address 10.0.2.15/241030gitlab # [ 8.494854] postgresql-pre-start[789]: performing post-bootstrap initialization ... ok1031gitlab # [ 8.811646] postgresql-pre-start[789]: syncing data to disk ... ok1032gitlab # [ 8.811975] postgresql-pre-start[789]: initdb: warning: enabling "trust" authentication for local connections1033gitlab # [ 8.813960] postgresql-pre-start[789]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.1034gitlab # [ 8.815987] postgresql-pre-start[789]: Success. You can now start the database server using:1035gitlab # [ 8.817273] postgresql-pre-start[789]: pg_ctl -D /var/lib/postgresql/18 -l logfile start1036gitlab # [ 8.966403] postgres[850]: [850] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit1037gitlab # [ 8.970507] postgres[850]: [850] LOG: listening on IPv6 address "::1", port 54321038gitlab # [ 8.973286] postgres[850]: [850] LOG: listening on IPv4 address "127.0.0.1", port 54321039gitlab # [ 8.976097] postgres[850]: [850] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"1040gitlab # [ 8.986540] postgres[859]: [859] LOG: database system was shut down at 2026-11-10 10:54:33 GMT1041gitlab # [ 8.994721] postgres[850]: [850] LOG: database system is ready to accept connections1042gitlab # [ 9.001347] systemd[1]: Started PostgreSQL Server.1043gitlab # [ 9.004316] systemd[1]: Starting PostgreSQL Setup Scripts...1044gitlab # [ 9.230249] postgresql-setup-start[874]: CREATE DATABASE1045gitlab # [ 9.238195] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[870]: mkdir -p /var/gitlab/state/home/.ssh: OK1046gitlab # [ 9.239670] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[870]: chmod 700 /var/gitlab/state/home/.ssh: OK1047gitlab # [ 9.252548] postgresql-setup-start[893]: CREATE ROLE1048gitlab # [ 9.260639] postgresql-setup-start[897]: ALTER DATABASE1049gitlab # [ 9.276591] systemd[1]: Finished gitlab-config.service.1050gitlab # [ 9.277712] postgresql-setup-start[907]: CREATE ROLE1051gitlab # [ 9.278758] systemd[1]: Started gitaly.service.1052gitlab # [ 9.279870] systemd[1]: Finished PostgreSQL Setup Scripts.1053gitlab # [ 9.281182] systemd[1]: Reached target PostgreSQL.1054gitlab # [ 9.282546] systemd[1]: Starting gitlab-db-config.service...1055gitlab # [ 9.283855] systemd[1]: Starting gitlab-postgresql.service...1056gitlab # [ 9.364691] gitlab-postgresql-start[920]: CREATE DATABASE1057gitlab # [ 9.407916] gitlab-postgresql-start[925]: CREATE EXTENSION1058gitlab # [ 9.483350] gitlab-postgresql-start[927]: CREATE EXTENSION1059gitlab # [ 9.486545] systemd[1]: Finished gitlab-postgresql.service.1060gitlab # [ 9.548651] dhcpcd[765]: eth0: soliciting an IPv6 router1061gitlab # [ 9.548826] dhcpcd[765]: eth0: Router Advertisement from fe80::21062gitlab # [ 9.552515] dhcpcd[765]: eth0: adding address fec0::5054:ff:fe12:3456/641063gitlab # [ 9.554692] dhcpcd[765]: eth0: adding route to fec0::/641064gitlab # [ 9.556275] dhcpcd[765]: eth0: adding default route via fe80::21065gitlab # [ 9.770067] gitaly[910]: time="2026-11-10T10:54:35.196Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=9101066gitlab # [ 9.770300] gitaly[910]: time="2026-11-10T10:54:35.196Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=9101067gitlab # [ 9.770655] gitaly[910]: time="2026-11-10T10:54:35.196Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=9101068gitlab # [ 9.771840] gitaly[910]: time="2026-11-10T10:54:35.196Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=9101069gitlab # [ 9.773049] gitaly[910]: time="2026-11-10T10:54:35.196Z" level=info msg="Starting Gitaly" pid=910 version=1070gitlab # [ 9.934900] gitaly[910]: time="2026-11-10T10:54:35.361Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=164 pid=9101071gitlab # [ 9.935360] gitaly[910]: time="2026-11-10T10:54:35.361Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=9101072gitlab # [ 9.936352] gitaly[910]: time="2026-11-10T10:54:35.362Z" level=info msg="finished initializing command factory" duration_ms=0 pid=9101073gitlab # [ 9.936967] gitaly[910]: time="2026-11-10T10:54:35.362Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-910/git-exec-2401696730.d/git pid=9101074gitlab # [ 9.954769] gitaly[910]: time="2026-11-10T10:54:35.381Z" level=info msg="finished detecting git version" duration_ms=19 pid=9101075gitlab # [ 9.954965] gitaly[910]: time="2026-11-10T10:54:35.381Z" level=info msg="using Git version" pid=910 version=2.53.g1076gitlab # [ 9.960675] gitaly[910]: time="2026-11-10T10:54:35.387Z" level=info msg="clearing disk cache object folder" pid=910 storage=default1077gitlab # [ 9.962528] gitaly[910]: time="2026-11-10T10:54:35.387Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4223573694 pid=910 storage=default1078gitlab # [ 9.965457] gitaly[910]: time="2026-11-10T10:54:35.387Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4223573694 pid=910 storage=default1079gitlab # [ 9.968588] gitaly[910]: time="2026-11-10T10:54:35.387Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=9101080gitlab # [ 9.968977] gitaly[910]: time="2026-11-10T10:54:35.387Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=9101081gitlab # [ 9.969666] gitaly[910]: time="2026-11-10T10:54:35.387Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache4223573694 clear_duration_ms=0 pid=910 storage=default1082gitlab # [ 10.001038] gitaly[910]: time="2026-11-10T10:54:35.427Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9101083gitlab # [ 10.003722] gitaly[910]: time="2026-11-10T10:54:35.427Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=910 schema=unix1084gitlab # [ 10.004085] gitaly[910]: time="2026-11-10T10:54:35.427Z" level=info msg="listening at address" address=/run/gitaly/gitaly-910/sock.d/intern pid=910 schema=unix1085gitlab # [ 10.004800] gitaly[910]: time="2026-11-10T10:54:35.428Z" level=info msg="finished tempdir cleaner walk" pid=910 storage=default time_ms=01086gitlab # [ 10.005595] gitaly[910]: time="2026-11-10T10:54:35.427Z" level=info msg="counting repositories" pid=9101087gitlab # [ 10.006111] gitaly[910]: time="2026-11-10T10:54:35.428Z" level=info msg="starting to count repositories" pid=910 storage_path=/var/gitlab/state/repositories1088gitlab # [ 10.006825] gitaly[910]: time="2026-11-10T10:54:35.428Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=910 storage_path=/var/gitlab/state/repositories1089gitlab # [ 10.007564] gitaly[910]: time="2026-11-10T10:54:35.428Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9101090gitlab # [ 10.008281] gitaly[910]: time="2026-11-10T10:54:35.428Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9101091gitlab # [ 10.009308] gitaly[910]: time="2026-11-10T10:54:35.429Z" level=info msg="Started Gitaly" duration_ms=232 pid=9101092gitlab # [ 10.010081] gitaly[910]: time="2026-11-10T10:54:35.429Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9101093gitlab # [ 10.010579] gitaly[910]: time="2026-11-10T10:54:35.429Z" level=info msg="maintenance: daily scheduled" pid=910 scheduled="2026-11-10 12:00:00 +0000 UTC"1094gitlab # [ 10.865795] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[915]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1095gitlab # [ 10.872344] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[915]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1096gitlab # [ 10.904550] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[915]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1097gitlab # [ 12.247586] gitaly[910]: time="2026-11-10T10:54:37.673Z" level=info msg="License database preloaded" duration_ms=2476 pid=9101098gitlab # [ 13.791285] dhcpcd[765]: eth0: leased 10.0.2.15 for 86400 seconds1099gitlab # [ 13.791679] dhcpcd[765]: eth0: adding route to 10.0.2.0/241100gitlab # [ 13.793591] dhcpcd[765]: eth0: adding default route via 10.0.2.21101gitlab # [ 13.918218] systemd[1]: Started DHCP Client.1102gitlab # [ 13.920584] systemd[1]: Reached target Network is Online.1103gitlab # [ 41.902891] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[915]: W, [2026-11-10T10:55:07.328771 #915] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1104gitlab # [ 43.852788] postgres[1326]: [1326] ERROR: relation "feature_gates" does not exist at character 5231105gitlab # [ 43.853410] postgres[1326]: [1326] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1106gitlab # [ 43.854073] postgres[1326]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1107gitlab # [ 43.854963] postgres[1326]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1108gitlab # [ 43.855995] postgres[1326]: attidentity AS identity,1109gitlab # [ 43.857073] postgres[1326]: attgenerated as attgenerated1110gitlab # [ 43.857933] postgres[1326]: FROM pg_attribute a1111gitlab # [ 43.858809] postgres[1326]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1112gitlab # [ 43.859720] postgres[1326]: LEFT JOIN pg_type t ON a.atttypid = t.oid1113gitlab # [ 43.860679] postgres[1326]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1114gitlab # [ 43.861701] postgres[1326]: WHERE a.attrelid = '"feature_gates"'::regclass1115gitlab # [ 43.862731] postgres[1326]: AND a.attnum > 0 AND NOT a.attisdropped1116gitlab # [ 43.863676] postgres[1326]: ORDER BY a.attnum1117gitlab # [ 43.864643] postgres[1326]:1118gitlab # [ 47.212683] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1359]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1119gitlab # [ 47.214705] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1359]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1120gitlab # [ 48.028438] gitlab-workhorse[823]: time="2026-11-10T10:55:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1121gitlab # [ 58.028533] gitlab-workhorse[823]: time="2026-11-10T10:55:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1122gitlab # [ 62.327897] postgres[1425]: [1425] ERROR: relation "table_without_models" does not exist1123gitlab # [ 62.330736] postgres[1425]: [1425] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1124gitlab # [ 68.038502] gitlab-workhorse[823]: time="2026-11-10T10:55:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1125gitlab # [ 73.691523] systemd[1]: Finished gitlab-db-config.service.1126gitlab # [ 73.695534] systemd[1]: Started gitlab-sidekiq.service.1127gitlab # [ 73.699724] systemd[1]: Starting gitlab.service...1128gitlab # [ 74.043672] bundle[1592]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1129gitlab # [ 74.044369] bundle[1592]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1130gitlab # [ 74.048339] bundle[1592]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1131gitlab # [ 76.999746] sidekiq-cluster[1591]: {"severity":"INFO","time":"2026-11-10T10:55:42.406Z","message":"Starting cluster with 1 processes"}1132gitlab # [ 77.068294] bundle[1592]: {"timestamp":"2026-11-10T10:55:42.472Z","pid":1592,"message":"* Enabling systemd notification integration"}1133gitlab # [ 77.069912] bundle[1592]: {"timestamp":"2026-11-10T10:55:42.494Z","pid":1592,"message":"Puma starting in cluster mode..."}1134gitlab # [ 77.071297] bundle[1592]: {"timestamp":"2026-11-10T10:55:42.494Z","pid":1592,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1135gitlab # [ 77.072796] bundle[1592]: {"timestamp":"2026-11-10T10:55:42.494Z","pid":1592,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1136gitlab # [ 77.074515] bundle[1592]: {"timestamp":"2026-11-10T10:55:42.494Z","pid":1592,"message":"* Min threads: 0"}1137gitlab # [ 77.075836] bundle[1592]: {"timestamp":"2026-11-10T10:55:42.494Z","pid":1592,"message":"* Max threads: 4"}1138gitlab # [ 77.077171] bundle[1592]: {"timestamp":"2026-11-10T10:55:42.494Z","pid":1592,"message":"* Environment: production"}1139gitlab # [ 77.078783] bundle[1592]: {"timestamp":"2026-11-10T10:55:42.494Z","pid":1592,"message":"* Master PID: 1592"}1140gitlab # [ 77.080338] bundle[1592]: {"timestamp":"2026-11-10T10:55:42.494Z","pid":1592,"message":"* Workers: 2"}1141gitlab # [ 77.081671] bundle[1592]: {"timestamp":"2026-11-10T10:55:42.494Z","pid":1592,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1142gitlab # [ 77.083380] bundle[1592]: {"timestamp":"2026-11-10T10:55:42.494Z","pid":1592,"message":"* Preloading application"}1143gitlab # [ 78.047987] gitlab-workhorse[823]: time="2026-11-10T10:55:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1144gitlab # [ 88.057091] gitlab-workhorse[823]: time="2026-11-10T10:55:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1145gitlab # [ 98.067755] gitlab-workhorse[823]: time="2026-11-10T10:56:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1146gitlab # [ 100.625725] bundle[1592]: W, [2026-11-10T10:56:06.051739 #1592] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1147gitlab # [ 103.648732] sidekiq-cluster[1622]: 2026-11-10T10:56:09.075Z pid=1622 tid=wla INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1622, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>431079424}1148gitlab # [ 103.835655] sidekiq-cluster[1622]: W, [2026-11-10T10:56:09.262123 #1622] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1149gitlab # [ 103.876399] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:09.302Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1150gitlab # [ 105.739232] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.165Z","message":"GitLab reliable fetch activated!"}1151gitlab # [ 105.739909] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.165Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1152gitlab # [ 105.761093] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.187Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1153gitlab # [ 105.763274] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.189Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1154gitlab # [ 105.765570] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.192Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1155gitlab # [ 105.769684] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.196Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1156gitlab # [ 105.773911] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.200Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1157gitlab # [ 105.777768] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.204Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1158gitlab # [ 105.781639] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.208Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1159gitlab # [ 105.784855] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.211Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1160gitlab # [ 105.788186] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.214Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1161gitlab # [ 105.791686] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.218Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1162gitlab # [ 105.795283] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.221Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1163gitlab # [ 105.800240] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.226Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1164gitlab # [ 105.803790] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.230Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1165gitlab # [ 105.807458] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.234Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1166gitlab # [ 105.811885] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.238Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1167gitlab # [ 105.815965] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.242Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1168gitlab # [ 105.823470] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.250Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1169gitlab # [ 105.829893] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.256Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1170gitlab # [ 105.836247] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.262Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1171gitlab # [ 105.842203] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.268Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1172gitlab # [ 105.848491] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.275Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1173gitlab # [ 105.854987] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.281Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1174gitlab # [ 105.860966] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.287Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1175gitlab # [ 105.868349] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.294Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1176gitlab # [ 105.874651] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.301Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1177gitlab # [ 105.881649] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.308Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1178gitlab # [ 105.887186] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.313Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1179gitlab # [ 105.892855] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.319Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1180gitlab # [ 105.898706] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.325Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1181gitlab # [ 105.904557] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.331Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1182gitlab # [ 105.909414] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.336Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1183gitlab # [ 105.915334] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.342Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1184gitlab # [ 105.920765] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.347Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1185gitlab # [ 105.926509] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.353Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1186gitlab # [ 105.932206] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.358Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1187gitlab # [ 105.937268] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.363Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1188gitlab # [ 105.942075] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.368Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1189gitlab # [ 105.947190] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.373Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1190gitlab # [ 105.952226] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.378Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1191gitlab # [ 105.956771] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.383Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1192gitlab # [ 105.962095] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.388Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1193gitlab # [ 105.966689] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.393Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1194gitlab # [ 105.971638] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.398Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1195gitlab # [ 105.976931] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.403Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1196gitlab # [ 105.982837] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.409Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1197gitlab # [ 105.987403] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.414Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1198gitlab # [ 105.992691] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.419Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1199gitlab # [ 105.997835] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.424Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1200gitlab # [ 106.003407] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.430Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1201gitlab # [ 106.008951] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.435Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1202gitlab # [ 106.014667] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.441Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1203gitlab # [ 106.018659] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.445Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1204gitlab # [ 106.028314] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.455Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1205gitlab # [ 106.030992] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.457Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1206gitlab # [ 106.033689] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.460Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1207gitlab # [ 106.038789] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.465Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1208gitlab # [ 106.043122] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.469Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1209gitlab # [ 106.048334] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.475Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1210gitlab # [ 106.052559] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.479Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1211gitlab # [ 106.056579] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.483Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1212gitlab # [ 106.060350] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.487Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1213gitlab # [ 106.064568] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.491Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1214gitlab # [ 106.068900] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.495Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1215gitlab # [ 106.073324] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.500Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1216gitlab # [ 106.079813] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.506Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1217gitlab # [ 106.127064] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.552Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1218gitlab # [ 106.131519] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.558Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1219gitlab # [ 106.138983] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.565Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1220gitlab # [ 106.148503] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.575Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1221gitlab # [ 106.157502] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.584Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1222gitlab # [ 106.167753] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.594Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1223gitlab # [ 106.176867] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.603Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1224gitlab # [ 106.185632] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.612Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1225gitlab # [ 106.193637] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.620Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1226gitlab # [ 106.202093] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.628Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1227gitlab # [ 106.209767] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.636Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1228gitlab # [ 106.217848] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.644Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1229gitlab # [ 106.225760] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.652Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1230gitlab # [ 106.232907] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.659Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1231gitlab # [ 106.241367] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.667Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1232gitlab # [ 106.249110] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.675Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1233gitlab # [ 106.257790] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.684Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1234gitlab # [ 106.266519] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.693Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1235gitlab # [ 106.275389] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.701Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1236gitlab # [ 106.284051] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.710Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1237gitlab # [ 106.292396] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.719Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1238gitlab # [ 106.301593] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.728Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1239gitlab # [ 106.310687] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.737Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1240gitlab # [ 106.318947] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.745Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1241gitlab # [ 106.326707] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.753Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1242gitlab # [ 106.331803] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.758Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1243gitlab # [ 106.356885] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.783Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1244gitlab # [ 106.364290] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.790Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1245gitlab # [ 106.373602] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.800Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1246gitlab # [ 106.381690] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.808Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1247gitlab # [ 106.391123] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.817Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1248gitlab # [ 106.402195] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.828Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1249gitlab # [ 106.411432] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.838Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1250gitlab # [ 106.430085] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.856Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1251gitlab # [ 106.434896] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.861Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1252gitlab # [ 106.440184] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.866Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1253gitlab # [ 106.447359] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.874Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1254gitlab # [ 106.450146] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.876Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1255gitlab # [ 106.452400] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.879Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1256gitlab # [ 106.454702] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.881Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1257gitlab # [ 106.457380] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.883Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1258gitlab # [ 106.460392] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.887Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1259gitlab # [ 106.463112] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.889Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1260gitlab # [ 106.467414] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:11.893Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1261gitlab # [ 108.068760] gitlab-workhorse[823]: time="2026-11-10T10:56:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1262gitlab # [ 109.199952] sidekiq-cluster[1622]: InitializerConnections Query: SET client_min_messages TO 'warning'1263gitlab # [ 109.201310] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1264gitlab # [ 109.202726] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1265gitlab # [ 109.204040] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1266gitlab # [ 109.205407] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1267gitlab # [ 109.206686] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1268gitlab # [ 109.208232] sidekiq-cluster[1622]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1269gitlab # [ 109.209829] sidekiq-cluster[1622]: InitializerConnections Query: SET standard_conforming_strings = on1270gitlab # [ 109.211089] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1271gitlab # [ 109.212502] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1272gitlab # [ 109.213806] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1273gitlab # [ 109.215095] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1274gitlab # [ 109.216326] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1275gitlab # [ 109.217822] sidekiq-cluster[1622]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1276gitlab # [ 109.219441] sidekiq-cluster[1622]: InitializerConnections Query: SET intervalstyle = iso_86011277gitlab # [ 109.220581] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1278gitlab # [ 109.221945] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1279gitlab # [ 109.223231] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1280gitlab # [ 109.224550] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1281gitlab # [ 109.225825] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1282gitlab # [ 109.227280] sidekiq-cluster[1622]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1283gitlab # [ 109.229067] sidekiq-cluster[1622]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname1284gitlab # [ 109.230978] sidekiq-cluster[1622]: FROM pg_type as t1285gitlab # [ 109.231720] sidekiq-cluster[1622]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz')1286gitlab # [ 109.233532] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1287gitlab # [ 109.234914] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1288gitlab # [ 109.236207] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1289gitlab # [ 109.237494] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1290gitlab # [ 109.238676] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1291gitlab # [ 109.240084] sidekiq-cluster[1622]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1292gitlab # [ 109.241572] sidekiq-cluster[1622]: InitializerConnections Query: SET SESSION timezone TO 'UTC'1293gitlab # [ 109.242757] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1294gitlab # [ 109.244163] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1295gitlab # [ 109.245421] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1296gitlab # [ 109.246807] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1297gitlab # [ 109.248066] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1298gitlab # [ 109.249472] sidekiq-cluster[1622]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1299gitlab # [ 109.251135] sidekiq-cluster[1622]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'per_thread_db_connection_pool_metrics'1300gitlab # [ 109.254417] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1301gitlab # [ 109.255836] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1302gitlab # [ 109.257066] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1303gitlab # [ 109.258436] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1304gitlab # [ 109.259692] sidekiq-cluster[1622]: InitializerConnections Backtrace: lib/gitlab/daemon.rb:58:in `block (2 levels) in start'1305gitlab # [ 109.261119] sidekiq-cluster[1622]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1306gitlab # [ 118.078409] gitlab-workhorse[823]: time="2026-11-10T10:56:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1307gitlab # [ 128.079772] gitlab-workhorse[823]: time="2026-11-10T10:56:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1308gitlab # [ 129.764707] postgres[2087]: [2087] ERROR: relation "table_without_models" does not exist1309gitlab # [ 129.765105] postgres[2087]: [2087] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1310gitlab # [ 129.938283] postgres[2097]: [2097] ERROR: relation "table_without_models" does not exist1311gitlab # [ 129.938775] postgres[2097]: [2097] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1312gitlab # [ 132.263765] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:37.690Z","message":"Booted Rails 7.2.3.1 application in production environment"}1313gitlab # [ 132.271879] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:37.698Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1314gitlab # [ 132.271995] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:37.698Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1315gitlab # [ 132.272805] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:37.698Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1316gitlab # [ 132.288251] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:37.714Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1317gitlab # [ 132.727286] bundle[1592]: {"timestamp":"2026-11-10T10:56:38.153Z","pid":1592,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1318gitlab # [ 132.727572] bundle[1592]: {"timestamp":"2026-11-10T10:56:38.153Z","pid":1592,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1319gitlab # [ 132.729261] bundle[1592]: {"timestamp":"2026-11-10T10:56:38.153Z","pid":1592,"message":"! #\u003cThread:0x00007977aa2fd910 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1320gitlab # [ 132.730746] bundle[1592]: {"timestamp":"2026-11-10T10:56:38.153Z","pid":1592,"message":"! #\u003cThread:0x00007977aa59c270 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1321gitlab # [ 132.732275] bundle[1592]: {"timestamp":"2026-11-10T10:56:38.153Z","pid":1592,"message":"Use Ctrl-C to stop"}1322gitlab # [ 132.733421] bundle[1592]: {"timestamp":"2026-11-10T10:56:38.153Z","pid":1592,"message":"Sending status to systemd every 1.0 sec"}1323gitlab # [ 133.125146] bundle[1592]: {"timestamp":"2026-11-10T10:56:38.550Z","pid":1592,"message":"- Worker 1 (PID: 2177) booted in 0.31s, phase: 0"}1324gitlab # [ 133.127814] bundle[1592]: {"timestamp":"2026-11-10T10:56:38.554Z","pid":1592,"message":"- Worker 0 (PID: 2173) booted in 0.34s, phase: 0"}1325gitlab # [ 133.134546] systemd[1]: Started gitlab.service.1326gitlab # [ 133.134741] systemd[1]: Reached target Common target for all GitLab services..1327gitlab # [ 133.138206] systemd[1]: Starting setup-gitlab.service...1328gitlab # [ 133.153784] setup-gitlab-start[2201]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1329gitlab # [ 133.602670] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:39.028Z","message":"Cleaning working queues"}1330gitlab # [ 133.606883] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:39.033Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"44f827b003c93acf6b4f5361","created_at":"2026-11-10T10:55:38.830Z","trace_propagation_headers":{"sentry-trace":"80d7423f307f42f59b5fa18de12a4f12-c174c2540ab84800","baggage":"sentry-trace_id=80d7423f307f42f59b5fa18de12a4f12,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"6b16f2d8f0f752a6d39cdf9f9874ce33","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T10:55:38.837Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-44f827b003c93acf6b4f5361: start","job_status":"start","queue_duration_s":60.196127,"scheduling_latency_s":60.196167}1331gitlab # [ 133.772830] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:56:39.199Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"44f827b003c93acf6b4f5361","created_at":"2026-11-10T10:55:38.830Z","trace_propagation_headers":{"sentry-trace":"80d7423f307f42f59b5fa18de12a4f12-c174c2540ab84800","baggage":"sentry-trace_id=80d7423f307f42f59b5fa18de12a4f12,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"6b16f2d8f0f752a6d39cdf9f9874ce33","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T10:55:38.837Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-44f827b003c93acf6b4f5361: done: 0.166042 sec","job_status":"done","queue_duration_s":60.196127,"scheduling_latency_s":60.196167,"redis_calls":26,"redis_duration_s":0.005657,"redis_read_bytes":914,"redis_write_bytes":2887,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.003421,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001383,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000853,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.08659257,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.166042,"completed_at":"2026-11-10T10:56:39.199Z","load_balancing_strategy":"primary","db_duration_s":0.058116,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1332gitlab # [ 135.408448] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:56:40 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 22331333gitlab # [ 135.414533] setup-gitlab-start[2201]: + TOKEN=nixbot-test-token-of-20-chars1334gitlab # [ 135.415318] setup-gitlab-start[2201]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1335gitlab # [ 135.428618] runuser[2226]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1336gitlab # [ 135.830871] setup-gitlab-start[2227]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1337gitlab # [ 135.831324] setup-gitlab-start[2227]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1338gitlab # [ 135.832516] setup-gitlab-start[2227]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1339gitlab # [ 136.422644] setup-gitlab-start[2227]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1340gitlab # [ 136.423235] setup-gitlab-start[2227]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1341gitlab # [ 136.431689] setup-gitlab-start[2227]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1342gitlab # [ 156.487594] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:01.913Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"670d018619d64fcc385c5bd0","created_at":"2026-11-10T10:57:01.863Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"07320b7d5ad882c18e8e18e10da376dd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:01.909Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-670d018619d64fcc385c5bd0: start","job_status":"start","queue_duration_s":0.003516,"scheduling_latency_s":0.003558}1343gitlab # [ 156.722141] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.148Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"670d018619d64fcc385c5bd0","created_at":"2026-11-10T10:57:01.863Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"07320b7d5ad882c18e8e18e10da376dd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:01.909Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-670d018619d64fcc385c5bd0: done: 0.234901 sec","job_status":"done","queue_duration_s":0.003516,"scheduling_latency_s":0.003558,"redis_calls":12,"redis_duration_s":0.007646,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00109,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003231,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003325,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001516673,"exclusive_lock_hold_duration_s":0.087847199,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.234901,"completed_at":"2026-11-10T10:57:02.147Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.041197,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 156.777655] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.151Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"1f651a1d814ba76140dda6bf","created_at":"2026-11-10T10:57:01.967Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11ec17beb84739e058908a2f6f5d30b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:01.978Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-1f651a1d814ba76140dda6bf: start","job_status":"start","queue_duration_s":0.17331,"scheduling_latency_s":0.17336}1345gitlab # [ 156.790621] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.181Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1346gitlab # [ 156.794246] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.207Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"1f651a1d814ba76140dda6bf","created_at":"2026-11-10T10:57:01.967Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11ec17beb84739e058908a2f6f5d30b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:01.978Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-1f651a1d814ba76140dda6bf: done: 0.055676 sec","job_status":"done","queue_duration_s":0.17331,"scheduling_latency_s":0.17336,"redis_calls":10,"redis_duration_s":0.00469,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002332,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049727,"duration_s":0.055676,"completed_at":"2026-11-10T10:57:02.207Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006643,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 156.831804] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.210Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6d7a3b286406d641e1353cb3","created_at":"2026-11-10T10:57:01.988Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38a5d8d5a2bea34629b92cf0bd5bf036","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.014Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6d7a3b286406d641e1353cb3: start","job_status":"start","queue_duration_s":0.195772,"scheduling_latency_s":0.195804}1348gitlab # [ 156.876156] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.302Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6d7a3b286406d641e1353cb3","created_at":"2026-11-10T10:57:01.988Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38a5d8d5a2bea34629b92cf0bd5bf036","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.014Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6d7a3b286406d641e1353cb3: done: 0.092548 sec","job_status":"done","queue_duration_s":0.195772,"scheduling_latency_s":0.195804,"redis_calls":16,"redis_duration_s":0.009825,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.006296,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002702,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000827,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024423,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000514032,"duration_s":0.092548,"completed_at":"2026-11-10T10:57:02.302Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035319,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 156.896782] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.304Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"3a7100ec5f1b5e66b7f7a799","created_at":"2026-11-10T10:57:02.043Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"32a3a194bf6235f0c1d7c276d20b6a53","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.051Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-3a7100ec5f1b5e66b7f7a799: start","job_status":"start","queue_duration_s":0.252753,"scheduling_latency_s":0.25278}1350gitlab # [ 156.954908] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.380Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"3a7100ec5f1b5e66b7f7a799","created_at":"2026-11-10T10:57:02.043Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"32a3a194bf6235f0c1d7c276d20b6a53","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.051Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-3a7100ec5f1b5e66b7f7a799: done: 0.076469 sec","job_status":"done","queue_duration_s":0.252753,"scheduling_latency_s":0.25278,"redis_calls":12,"redis_duration_s":0.00498,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000673,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002069,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002238,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001238426,"exclusive_lock_hold_duration_s":0.009966909,"duration_s":0.076469,"completed_at":"2026-11-10T10:57:02.380Z","load_balancing_strategy":"primary","db_duration_s":0.040237,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351gitlab # [ 156.958172] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.384Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"5d39c97b03a794d3ff4ccaba","created_at":"2026-11-10T10:57:02.056Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"814d1ebc8d3abb7618ef0d39532ce387","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.062Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-5d39c97b03a794d3ff4ccaba: start","job_status":"start","queue_duration_s":0.322365,"scheduling_latency_s":0.322393}1352gitlab # [ 156.973280] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.400Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1353gitlab # [ 156.978701] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.405Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"5d39c97b03a794d3ff4ccaba","created_at":"2026-11-10T10:57:02.056Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"814d1ebc8d3abb7618ef0d39532ce387","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.062Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-5d39c97b03a794d3ff4ccaba: done: 0.020686 sec","job_status":"done","queue_duration_s":0.322365,"scheduling_latency_s":0.322393,"redis_calls":10,"redis_duration_s":0.001625,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00031,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000362,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000254781,"duration_s":0.020686,"completed_at":"2026-11-10T10:57:02.405Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000451,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354gitlab # [ 157.040751] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.410Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ddcbe4d1985811c016008aba","created_at":"2026-11-10T10:57:02.065Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eab1f9d4296e1ff4d03c357aee6c3a43","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T10:57:02.069Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ddcbe4d1985811c016008aba: start","job_status":"start","queue_duration_s":0.341096,"scheduling_latency_s":0.341123}1355gitlab # [ 157.041466] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.421Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1356gitlab # [ 157.042111] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.426Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"ddcbe4d1985811c016008aba","created_at":"2026-11-10T10:57:02.065Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eab1f9d4296e1ff4d03c357aee6c3a43","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-11-10T10:57:02.069Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-ddcbe4d1985811c016008aba: done: 0.015993 sec","job_status":"done","queue_duration_s":0.341096,"scheduling_latency_s":0.341123,"redis_calls":10,"redis_duration_s":0.0019529999999999999,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000319,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001217,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000417,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000212876,"duration_s":0.015993,"completed_at":"2026-11-10T10:57:02.426Z","load_balancing_strategy":"primary","db_duration_s":0.000302,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357gitlab # [ 157.085245] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.427Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"eb9a4e9012ccf7320c2d9cdc","created_at":"2026-11-10T10:57:02.073Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"96d7ad86f7a41d3df05f7c2b90b3e0cd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.081Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-eb9a4e9012ccf7320c2d9cdc: start","job_status":"start","queue_duration_s":0.345153,"scheduling_latency_s":0.345168}1358gitlab # [ 157.085873] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.450Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"eb9a4e9012ccf7320c2d9cdc","created_at":"2026-11-10T10:57:02.073Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"96d7ad86f7a41d3df05f7c2b90b3e0cd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.081Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-eb9a4e9012ccf7320c2d9cdc: done: 0.023041 sec","job_status":"done","queue_duration_s":0.345153,"scheduling_latency_s":0.345168,"redis_calls":19,"redis_duration_s":0.00234,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000494,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000226,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001201,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000419,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002288,"duration_s":0.023041,"completed_at":"2026-11-10T10:57:02.450Z","load_balancing_strategy":"primary","db_duration_s":0.001478,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359gitlab # [ 157.133577] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.451Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0436e1f62363d6cc06408818","created_at":"2026-11-10T10:57:02.087Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab417954737990c0869e6147bfdf8283","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.093Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0436e1f62363d6cc06408818: start","job_status":"start","queue_duration_s":0.358091,"scheduling_latency_s":0.358106}1360gitlab # [ 157.134414] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.474Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0436e1f62363d6cc06408818","created_at":"2026-11-10T10:57:02.087Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab417954737990c0869e6147bfdf8283","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.093Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0436e1f62363d6cc06408818: done: 0.023764 sec","job_status":"done","queue_duration_s":0.358091,"scheduling_latency_s":0.358106,"redis_calls":10,"redis_duration_s":0.001265,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000236,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000218185,"duration_s":0.023764,"completed_at":"2026-11-10T10:57:02.474Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002361,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361gitlab # [ 157.191656] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f044f5c3f05bcd1bbf7ca158","created_at":"2026-11-10T10:57:02.100Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e40da5c87a7503a33eb980bdd3d0832d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.105Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f044f5c3f05bcd1bbf7ca158: start","job_status":"start","queue_duration_s":0.371019,"scheduling_latency_s":0.371034}1362gitlab # [ 157.192466] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.488Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1363gitlab # [ 157.192930] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.495Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f044f5c3f05bcd1bbf7ca158","created_at":"2026-11-10T10:57:02.100Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e40da5c87a7503a33eb980bdd3d0832d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.105Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f044f5c3f05bcd1bbf7ca158: done: 0.019161 sec","job_status":"done","queue_duration_s":0.371019,"scheduling_latency_s":0.371034,"redis_calls":10,"redis_duration_s":0.00414,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001084,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002339,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000717,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001018007,"duration_s":0.019161,"completed_at":"2026-11-10T10:57:02.495Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002452,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 157.240686] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.498Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"dc3936e1fb72dd78c394ab4c","created_at":"2026-11-10T10:57:02.145Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c37093b62ad81b0d98fb9347e753f5c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T10:57:02.157Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-dc3936e1fb72dd78c394ab4c: start","job_status":"start","queue_duration_s":0.341216,"scheduling_latency_s":0.341232}1365gitlab # [ 157.241509] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.530Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"dc3936e1fb72dd78c394ab4c","created_at":"2026-11-10T10:57:02.145Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c37093b62ad81b0d98fb9347e753f5c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-11-10T10:57:02.157Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-dc3936e1fb72dd78c394ab4c: done: 0.031882 sec","job_status":"done","queue_duration_s":0.341216,"scheduling_latency_s":0.341232,"redis_calls":10,"redis_duration_s":0.003483,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000612,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002578,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000293,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013547,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000239137,"duration_s":0.031882,"completed_at":"2026-11-10T10:57:02.530Z","load_balancing_strategy":"primary","db_duration_s":0.009168,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 157.287133] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.531Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c49580652cd250cbb9c6b3c9","created_at":"2026-11-10T10:57:02.180Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.214Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c49580652cd250cbb9c6b3c9: start","job_status":"start","queue_duration_s":0.316523,"scheduling_latency_s":0.31654}1367gitlab # [ 157.287814] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.598Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c49580652cd250cbb9c6b3c9","created_at":"2026-11-10T10:57:02.180Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.214Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c49580652cd250cbb9c6b3c9: done: 0.067537 sec","job_status":"done","queue_duration_s":0.316523,"scheduling_latency_s":0.31654,"redis_calls":71,"redis_duration_s":0.009944,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001296,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000618,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.00766,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00037,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000218184,"duration_s":0.067537,"completed_at":"2026-11-10T10:57:02.598Z","load_balancing_strategy":"primary","db_duration_s":0.001107,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 157.332897] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.600Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"5e9a75ed2a3c85ec5d3b52c3","created_at":"2026-11-10T10:57:02.239Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab56d9f9df215c5406211064385a316b","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.249Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-5e9a75ed2a3c85ec5d3b52c3: start","job_status":"start","queue_duration_s":0.350757,"scheduling_latency_s":0.350773}1369gitlab # [ 157.333640] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.632Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"5e9a75ed2a3c85ec5d3b52c3","created_at":"2026-11-10T10:57:02.239Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab56d9f9df215c5406211064385a316b","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.249Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-5e9a75ed2a3c85ec5d3b52c3: done: 0.032748 sec","job_status":"done","queue_duration_s":0.350757,"scheduling_latency_s":0.350773,"redis_calls":10,"redis_duration_s":0.0055390000000000005,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004478,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000812,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027685,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.032748,"completed_at":"2026-11-10T10:57:02.632Z","load_balancing_strategy":"primary","db_duration_s":0.012591,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 157.377033] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.633Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"f82f91bfb8ce95a5c5c421ee","created_at":"2026-11-10T10:57:02.254Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef17421fa1d064753e827fc7dee197dc","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.260Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-f82f91bfb8ce95a5c5c421ee: start","job_status":"start","queue_duration_s":0.37368,"scheduling_latency_s":0.373723}1371gitlab # [ 157.377737] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.650Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"f82f91bfb8ce95a5c5c421ee","created_at":"2026-11-10T10:57:02.254Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef17421fa1d064753e827fc7dee197dc","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.260Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-f82f91bfb8ce95a5c5c421ee: done: 0.016573 sec","job_status":"done","queue_duration_s":0.37368,"scheduling_latency_s":0.373723,"redis_calls":12,"redis_duration_s":0.002338,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000722,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000733,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000543924,"exclusive_lock_hold_duration_s":0.002229614,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.016573,"completed_at":"2026-11-10T10:57:02.650Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001109,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 157.421800] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.651Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"80c520011eba6542e8d5a8e3","created_at":"2026-11-10T10:57:02.272Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a77475ca945faf59e91c3f480843cd6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.280Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-80c520011eba6542e8d5a8e3: start","job_status":"start","queue_duration_s":0.3705,"scheduling_latency_s":0.370515}1373gitlab # [ 157.425850] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.666Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"80c520011eba6542e8d5a8e3","created_at":"2026-11-10T10:57:02.272Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a77475ca945faf59e91c3f480843cd6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.280Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-80c520011eba6542e8d5a8e3: done: 0.015103 sec","job_status":"done","queue_duration_s":0.3705,"scheduling_latency_s":0.370515,"redis_calls":12,"redis_duration_s":0.001729,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000712,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000253,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008587,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000203099,"duration_s":0.015103,"completed_at":"2026-11-10T10:57:02.666Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000532,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 157.476373] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.667Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3bc5e79262e99a9a63fdc610","created_at":"2026-11-10T10:57:02.298Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d59207643fe64768912027426d4899ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.307Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3bc5e79262e99a9a63fdc610: start","job_status":"start","queue_duration_s":0.360106,"scheduling_latency_s":0.360121}1375gitlab # [ 157.478240] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.681Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"3bc5e79262e99a9a63fdc610","created_at":"2026-11-10T10:57:02.298Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d59207643fe64768912027426d4899ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.307Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-3bc5e79262e99a9a63fdc610: done: 0.014374 sec","job_status":"done","queue_duration_s":0.360106,"scheduling_latency_s":0.360121,"redis_calls":10,"redis_duration_s":0.001287,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000257,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205333,"duration_s":0.014374,"completed_at":"2026-11-10T10:57:02.681Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002183,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 157.524902] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.682Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"b1e48b8d8442999f82817901","created_at":"2026-11-10T10:57:02.336Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5fd1790f9bf6968761be448bd05e51d2","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.340Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-b1e48b8d8442999f82817901: start","job_status":"start","queue_duration_s":0.342414,"scheduling_latency_s":0.342428}1377gitlab # [ 157.568094] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.702Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"b1e48b8d8442999f82817901","created_at":"2026-11-10T10:57:02.336Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5fd1790f9bf6968761be448bd05e51d2","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.340Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-b1e48b8d8442999f82817901: done: 0.019984 sec","job_status":"done","queue_duration_s":0.342414,"scheduling_latency_s":0.342428,"redis_calls":13,"redis_duration_s":0.001596,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00024,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000814,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000542,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00601026,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.019984,"completed_at":"2026-11-10T10:57:02.702Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001587,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1378gitlab # [ 157.571297] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.703Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"c005845ac9c451a899b27551","created_at":"2026-11-10T10:57:02.344Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31cfdfe6b90ab7e9151fb9ed37eaba0f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T10:57:02.347Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-c005845ac9c451a899b27551: start","job_status":"start","queue_duration_s":0.356367,"scheduling_latency_s":0.35638}1379gitlab # [ 157.583822] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.711Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1380gitlab # [ 157.587301] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.716Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"c005845ac9c451a899b27551","created_at":"2026-11-10T10:57:02.344Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31cfdfe6b90ab7e9151fb9ed37eaba0f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-11-10T10:57:02.347Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-c005845ac9c451a899b27551: done: 0.012282 sec","job_status":"done","queue_duration_s":0.356367,"scheduling_latency_s":0.35638,"redis_calls":10,"redis_duration_s":0.001165,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000659,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000225,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000199466,"duration_s":0.012282,"completed_at":"2026-11-10T10:57:02.715Z","load_balancing_strategy":"primary","db_duration_s":0.000256,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1381gitlab # [ 157.628118] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.717Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"3fdefc98c6373e1ffc1282db","created_at":"2026-11-10T10:57:02.355Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c222e1f15d11751aa136854e3a040acb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.364Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-3fdefc98c6373e1ffc1282db: start","job_status":"start","queue_duration_s":0.352401,"scheduling_latency_s":0.352414}1382gitlab # [ 157.668860] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.754Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"3fdefc98c6373e1ffc1282db","created_at":"2026-11-10T10:57:02.355Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c222e1f15d11751aa136854e3a040acb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.364Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-3fdefc98c6373e1ffc1282db: done: 0.03737 sec","job_status":"done","queue_duration_s":0.352401,"scheduling_latency_s":0.352414,"redis_calls":14,"redis_duration_s":0.001822,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000496,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000718,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000608,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000319873,"exclusive_lock_hold_duration_s":0.022576892,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.03737,"completed_at":"2026-11-10T10:57:02.754Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017047,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1383gitlab # [ 157.678116] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.755Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"55f3a3bf41ccad9982a8dbf8","created_at":"2026-11-10T10:57:02.437Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-06bf1b4c4ed04b5b","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"96d7ad86f7a41d3df05f7c2b90b3e0cd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.445Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-55f3a3bf41ccad9982a8dbf8: start","job_status":"start","queue_duration_s":0.310424,"scheduling_latency_s":0.310446}1384gitlab # [ 157.720890] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.769Z","message":"No existing merge request to be cleaned up."}1385gitlab # [ 157.721555] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.778Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"55f3a3bf41ccad9982a8dbf8","created_at":"2026-11-10T10:57:02.437Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-06bf1b4c4ed04b5b","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"96d7ad86f7a41d3df05f7c2b90b3e0cd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.445Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-55f3a3bf41ccad9982a8dbf8: done: 0.022629 sec","job_status":"done","queue_duration_s":0.310424,"scheduling_latency_s":0.310446,"redis_calls":13,"redis_duration_s":0.001902,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000263,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000681,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000227403,"duration_s":0.022629,"completed_at":"2026-11-10T10:57:02.778Z","load_balancing_strategy":"primary","db_duration_s":0.004309,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 157.763632] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.779Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"685cbcfeb6046ac79e7f24c0","created_at":"2026-11-10T10:57:02.437Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-06bf1b4c4ed04b5b","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"96d7ad86f7a41d3df05f7c2b90b3e0cd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.445Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-685cbcfeb6046ac79e7f24c0: start","job_status":"start","queue_duration_s":0.334025,"scheduling_latency_s":0.33404}1387gitlab # [ 157.764469] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.786Z","message":"No existing merge request to be cleaned up."}1388gitlab # [ 157.764990] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.791Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"685cbcfeb6046ac79e7f24c0","created_at":"2026-11-10T10:57:02.437Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-06bf1b4c4ed04b5b","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"96d7ad86f7a41d3df05f7c2b90b3e0cd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.445Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-685cbcfeb6046ac79e7f24c0: done: 0.01226 sec","job_status":"done","queue_duration_s":0.334025,"scheduling_latency_s":0.33404,"redis_calls":10,"redis_duration_s":0.001213,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000698,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01226,"completed_at":"2026-11-10T10:57:02.791Z","load_balancing_strategy":"primary","db_duration_s":0.001328,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389gitlab # [ 157.806648] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.792Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"07e8494d621e903820af5652","created_at":"2026-11-10T10:57:02.437Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-06bf1b4c4ed04b5b","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"96d7ad86f7a41d3df05f7c2b90b3e0cd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.445Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-07e8494d621e903820af5652: start","job_status":"start","queue_duration_s":0.347274,"scheduling_latency_s":0.34729}1390gitlab # [ 157.807523] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.799Z","message":"No existing merge request to be cleaned up."}1391gitlab # [ 157.808067] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.804Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"07e8494d621e903820af5652","created_at":"2026-11-10T10:57:02.437Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-06bf1b4c4ed04b5b","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"96d7ad86f7a41d3df05f7c2b90b3e0cd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.445Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-07e8494d621e903820af5652: done: 0.011749 sec","job_status":"done","queue_duration_s":0.347274,"scheduling_latency_s":0.34729,"redis_calls":10,"redis_duration_s":0.0011970000000000001,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000478,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000719,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011749,"completed_at":"2026-11-10T10:57:02.804Z","load_balancing_strategy":"primary","db_duration_s":0.000949,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392gitlab # [ 157.821776] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.805Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"553a56dd0b28a8804536c89d","created_at":"2026-11-10T10:57:02.437Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-06bf1b4c4ed04b5b","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"96d7ad86f7a41d3df05f7c2b90b3e0cd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.445Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-553a56dd0b28a8804536c89d: start","job_status":"start","queue_duration_s":0.360105,"scheduling_latency_s":0.360121}1393gitlab # [ 157.863295] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.812Z","message":"No existing merge request to be cleaned up."}1394gitlab # [ 157.864064] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.817Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"553a56dd0b28a8804536c89d","created_at":"2026-11-10T10:57:02.437Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-06bf1b4c4ed04b5b","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"96d7ad86f7a41d3df05f7c2b90b3e0cd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.445Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-553a56dd0b28a8804536c89d: done: 0.012013 sec","job_status":"done","queue_duration_s":0.360105,"scheduling_latency_s":0.360121,"redis_calls":10,"redis_duration_s":0.001176,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000485,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000691,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012013,"completed_at":"2026-11-10T10:57:02.817Z","load_balancing_strategy":"primary","db_duration_s":0.001026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1395gitlab # [ 157.869735] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.818Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ce90ea835ec96a7b5ceba545","created_at":"2026-11-10T10:57:02.381Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"edb8006698f99119babe74e26f2d0073","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.476Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ce90ea835ec96a7b5ceba545: start","job_status":"start","queue_duration_s":0.341687,"scheduling_latency_s":0.341703}1396gitlab # [ 157.882975] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.826Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1397gitlab # [ 157.886182] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.830Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ce90ea835ec96a7b5ceba545","created_at":"2026-11-10T10:57:02.381Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"edb8006698f99119babe74e26f2d0073","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.476Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ce90ea835ec96a7b5ceba545: done: 0.012809 sec","job_status":"done","queue_duration_s":0.341687,"scheduling_latency_s":0.341703,"redis_calls":10,"redis_duration_s":0.001139,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000235,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00022,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208685,"duration_s":0.012809,"completed_at":"2026-11-10T10:57:02.830Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000274,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 157.949455] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:02.831Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"48a8d1de221f7e64c7b8f313","created_at":"2026-11-10T10:57:02.484Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"210a08328720857852be8136ff89b1d7","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.491Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-48a8d1de221f7e64c7b8f313: start","job_status":"start","queue_duration_s":0.340287,"scheduling_latency_s":0.3403}1399gitlab # [ 157.950219] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.096Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"48a8d1de221f7e64c7b8f313","created_at":"2026-11-10T10:57:02.484Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"210a08328720857852be8136ff89b1d7","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.491Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-48a8d1de221f7e64c7b8f313: done: 0.264638 sec","job_status":"done","queue_duration_s":0.340287,"scheduling_latency_s":0.3403,"redis_calls":565,"redis_duration_s":0.073044,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00023,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.072112,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.212546,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000227962,"duration_s":0.264638,"completed_at":"2026-11-10T10:57:03.096Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000266,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 157.997697] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.097Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9908a24848a3b92160e8982f","created_at":"2026-11-10T10:57:02.498Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"305327ab4493179a0536366b052ae557","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.505Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9908a24848a3b92160e8982f: start","job_status":"start","queue_duration_s":0.591608,"scheduling_latency_s":0.591624}1401gitlab # [ 157.998457] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.233Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9908a24848a3b92160e8982f","created_at":"2026-11-10T10:57:02.498Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-548fd7f96d504402","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"305327ab4493179a0536366b052ae557","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.505Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9908a24848a3b92160e8982f: done: 0.136162 sec","job_status":"done","queue_duration_s":0.591608,"scheduling_latency_s":0.591624,"redis_calls":12,"redis_duration_s":0.001737,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000261,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000698,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.10342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000358146,"exclusive_lock_hold_duration_s":0.121771622,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.136162,"completed_at":"2026-11-10T10:57:03.233Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.036171,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 158.053030] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6decfd94b34e9e9c3d753d9d","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6decfd94b34e9e9c3d753d9d: start","job_status":"start","queue_duration_s":0.642492,"scheduling_latency_s":0.642508}1403gitlab # [ 158.053747] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6decfd94b34e9e9c3d753d9d","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6decfd94b34e9e9c3d753d9d: done: 0.015472 sec","job_status":"done","queue_duration_s":0.642492,"scheduling_latency_s":0.642508,"redis_calls":16,"redis_duration_s":0.0020800000000000003,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000603,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001222,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000202539,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015472,"completed_at":"2026-11-10T10:57:03.250Z","load_balancing_strategy":"primary","db_duration_s":0.000607,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 158.098689] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c62b286d487c19148bb3561","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c62b286d487c19148bb3561: start","job_status":"start","queue_duration_s":0.658861,"scheduling_latency_s":0.658875}1405gitlab # [ 158.099677] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c62b286d487c19148bb3561","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c62b286d487c19148bb3561: done: 0.009569 sec","job_status":"done","queue_duration_s":0.658861,"scheduling_latency_s":0.658875,"redis_calls":13,"redis_duration_s":0.001385,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000442,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009569,"completed_at":"2026-11-10T10:57:03.260Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 158.142939] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca232fb73c67eb6a5273519d","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca232fb73c67eb6a5273519d: start","job_status":"start","queue_duration_s":0.669241,"scheduling_latency_s":0.669254}1407gitlab # [ 158.143694] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca232fb73c67eb6a5273519d","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca232fb73c67eb6a5273519d: done: 0.009543 sec","job_status":"done","queue_duration_s":0.669241,"scheduling_latency_s":0.669254,"redis_calls":13,"redis_duration_s":0.001503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000456,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009543,"completed_at":"2026-11-10T10:57:03.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 158.201814] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"447eda80e4aa817b05646924","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-447eda80e4aa817b05646924: start","job_status":"start","queue_duration_s":0.679593,"scheduling_latency_s":0.679607}1409gitlab # [ 158.202613] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"447eda80e4aa817b05646924","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-447eda80e4aa817b05646924: done: 0.009294 sec","job_status":"done","queue_duration_s":0.679593,"scheduling_latency_s":0.679607,"redis_calls":13,"redis_duration_s":0.001348,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000439,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009294,"completed_at":"2026-11-10T10:57:03.281Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 158.251625] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc85621b6b62be1e6c3308b6","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc85621b6b62be1e6c3308b6: start","job_status":"start","queue_duration_s":0.689664,"scheduling_latency_s":0.689678}1411gitlab # [ 158.252487] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.291Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc85621b6b62be1e6c3308b6","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc85621b6b62be1e6c3308b6: done: 0.00927 sec","job_status":"done","queue_duration_s":0.689664,"scheduling_latency_s":0.689678,"redis_calls":13,"redis_duration_s":0.001429,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000927,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005913,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00927,"completed_at":"2026-11-10T10:57:03.291Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 158.301172] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"286c0bcae5dd7db9f4280f39","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-286c0bcae5dd7db9f4280f39: start","job_status":"start","queue_duration_s":0.700111,"scheduling_latency_s":0.700127}1413gitlab # [ 158.301908] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"286c0bcae5dd7db9f4280f39","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-286c0bcae5dd7db9f4280f39: done: 0.009698 sec","job_status":"done","queue_duration_s":0.700111,"scheduling_latency_s":0.700127,"redis_calls":13,"redis_duration_s":0.001595,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009698,"completed_at":"2026-11-10T10:57:03.302Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414gitlab # [ 158.342107] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70a9beda48b1b750259e3ffb","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70a9beda48b1b750259e3ffb: start","job_status":"start","queue_duration_s":0.710681,"scheduling_latency_s":0.710695}1415gitlab # [ 158.342702] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.312Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70a9beda48b1b750259e3ffb","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70a9beda48b1b750259e3ffb: done: 0.009712 sec","job_status":"done","queue_duration_s":0.710681,"scheduling_latency_s":0.710695,"redis_calls":13,"redis_duration_s":0.0015860000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009712,"completed_at":"2026-11-10T10:57:03.312Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 158.382963] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbc5f97d7a6b039c0b0f0992","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbc5f97d7a6b039c0b0f0992: start","job_status":"start","queue_duration_s":0.721258,"scheduling_latency_s":0.721272}1417gitlab # [ 158.383727] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbc5f97d7a6b039c0b0f0992","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbc5f97d7a6b039c0b0f0992: done: 0.0098 sec","job_status":"done","queue_duration_s":0.721258,"scheduling_latency_s":0.721272,"redis_calls":13,"redis_duration_s":0.001642,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0098,"completed_at":"2026-11-10T10:57:03.323Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418gitlab # [ 158.422708] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e78cd86022a717e01f91595c","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e78cd86022a717e01f91595c: start","job_status":"start","queue_duration_s":0.731929,"scheduling_latency_s":0.731944}1419gitlab # [ 158.423512] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e78cd86022a717e01f91595c","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e78cd86022a717e01f91595c: done: 0.010166 sec","job_status":"done","queue_duration_s":0.731929,"scheduling_latency_s":0.731944,"redis_calls":13,"redis_duration_s":0.0016029999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010166,"completed_at":"2026-11-10T10:57:03.334Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420gitlab # [ 158.463790] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8f0010e95edccaed995fed1","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8f0010e95edccaed995fed1: start","job_status":"start","queue_duration_s":0.742982,"scheduling_latency_s":0.742995}1421gitlab # [ 158.464541] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8f0010e95edccaed995fed1","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8f0010e95edccaed995fed1: done: 0.009858 sec","job_status":"done","queue_duration_s":0.742982,"scheduling_latency_s":0.742995,"redis_calls":13,"redis_duration_s":0.0016769999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006374,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009858,"completed_at":"2026-11-10T10:57:03.345Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 158.504022] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5dd60da85bb3d155ee297316","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5dd60da85bb3d155ee297316: start","job_status":"start","queue_duration_s":0.753715,"scheduling_latency_s":0.753729}1423gitlab # [ 158.504762] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5dd60da85bb3d155ee297316","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5dd60da85bb3d155ee297316: done: 0.009614 sec","job_status":"done","queue_duration_s":0.753715,"scheduling_latency_s":0.753729,"redis_calls":13,"redis_duration_s":0.001568,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009614,"completed_at":"2026-11-10T10:57:03.355Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 158.544304] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34e6b5bdb4568c3f4747659a","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34e6b5bdb4568c3f4747659a: start","job_status":"start","queue_duration_s":0.764177,"scheduling_latency_s":0.764191}1425gitlab # [ 158.549453] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34e6b5bdb4568c3f4747659a","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34e6b5bdb4568c3f4747659a: done: 0.009728 sec","job_status":"done","queue_duration_s":0.764177,"scheduling_latency_s":0.764191,"redis_calls":13,"redis_duration_s":0.00168,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000647,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009728,"completed_at":"2026-11-10T10:57:03.366Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426gitlab # [ 158.584573] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"17167e17d9c954244dd88f2c","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-17167e17d9c954244dd88f2c: start","job_status":"start","queue_duration_s":0.775074,"scheduling_latency_s":0.775088}1427gitlab # [ 158.624173] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"17167e17d9c954244dd88f2c","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-17167e17d9c954244dd88f2c: done: 0.00966 sec","job_status":"done","queue_duration_s":0.775074,"scheduling_latency_s":0.775088,"redis_calls":13,"redis_duration_s":0.001581,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00966,"completed_at":"2026-11-10T10:57:03.377Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428gitlab # [ 158.629567] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80f4a2e5b218f8bd67f4e662","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80f4a2e5b218f8bd67f4e662: start","job_status":"start","queue_duration_s":0.785587,"scheduling_latency_s":0.785602}1429gitlab # [ 158.640696] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80f4a2e5b218f8bd67f4e662","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80f4a2e5b218f8bd67f4e662: done: 0.009651 sec","job_status":"done","queue_duration_s":0.785587,"scheduling_latency_s":0.785602,"redis_calls":13,"redis_duration_s":0.001611,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009651,"completed_at":"2026-11-10T10:57:03.387Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430gitlab # [ 158.691768] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69b8c688b30ea2b4665d499e","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69b8c688b30ea2b4665d499e: start","job_status":"start","queue_duration_s":0.796069,"scheduling_latency_s":0.796083}1431gitlab # [ 158.692648] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69b8c688b30ea2b4665d499e","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69b8c688b30ea2b4665d499e: done: 0.009677 sec","job_status":"done","queue_duration_s":0.796069,"scheduling_latency_s":0.796083,"redis_calls":13,"redis_duration_s":0.001615,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006206,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009677,"completed_at":"2026-11-10T10:57:03.398Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 158.721534] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c80139de9f6ee084ba639cb","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c80139de9f6ee084ba639cb: start","job_status":"start","queue_duration_s":0.806621,"scheduling_latency_s":0.806635}1433gitlab # [ 158.732261] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c80139de9f6ee084ba639cb","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c80139de9f6ee084ba639cb: done: 0.009924 sec","job_status":"done","queue_duration_s":0.806621,"scheduling_latency_s":0.806635,"redis_calls":13,"redis_duration_s":0.001598,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009924,"completed_at":"2026-11-10T10:57:03.408Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1434gitlab # [ 158.774972] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f024c50663b8e925069571d0","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f024c50663b8e925069571d0: start","job_status":"start","queue_duration_s":0.8174,"scheduling_latency_s":0.817414}1435gitlab # [ 158.813090] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.419Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f024c50663b8e925069571d0","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f024c50663b8e925069571d0: done: 0.009668 sec","job_status":"done","queue_duration_s":0.8174,"scheduling_latency_s":0.817414,"redis_calls":13,"redis_duration_s":0.0015710000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009668,"completed_at":"2026-11-10T10:57:03.419Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436gitlab # [ 158.814117] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b93325c9f050a3e111b8cd9b","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b93325c9f050a3e111b8cd9b: start","job_status":"start","queue_duration_s":0.827939,"scheduling_latency_s":0.827952}1437gitlab # [ 158.859944] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b93325c9f050a3e111b8cd9b","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b93325c9f050a3e111b8cd9b: done: 0.011745 sec","job_status":"done","queue_duration_s":0.827939,"scheduling_latency_s":0.827952,"redis_calls":13,"redis_duration_s":0.0036890000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011745,"completed_at":"2026-11-10T10:57:03.431Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1438gitlab # [ 158.861404] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"902fc6459ad34e46cef6a2ce","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-902fc6459ad34e46cef6a2ce: start","job_status":"start","queue_duration_s":0.840543,"scheduling_latency_s":0.840582}1439gitlab # [ 158.868598] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"902fc6459ad34e46cef6a2ce","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-902fc6459ad34e46cef6a2ce: done: 0.009986 sec","job_status":"done","queue_duration_s":0.840543,"scheduling_latency_s":0.840582,"redis_calls":13,"redis_duration_s":0.00174,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009986,"completed_at":"2026-11-10T10:57:03.442Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440gitlab # [ 158.920197] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fac7cd71262134e6f98526c","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fac7cd71262134e6f98526c: start","job_status":"start","queue_duration_s":0.851791,"scheduling_latency_s":0.851805}1441gitlab # [ 158.921111] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.454Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fac7cd71262134e6f98526c","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fac7cd71262134e6f98526c: done: 0.010102 sec","job_status":"done","queue_duration_s":0.851791,"scheduling_latency_s":0.851805,"redis_calls":13,"redis_duration_s":0.001604,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001122,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010102,"completed_at":"2026-11-10T10:57:03.454Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442gitlab # [ 158.959210] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c719bc5bd11fe5b40bf8022b","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c719bc5bd11fe5b40bf8022b: start","job_status":"start","queue_duration_s":0.86271,"scheduling_latency_s":0.862723}1443gitlab # [ 158.960096] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c719bc5bd11fe5b40bf8022b","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c719bc5bd11fe5b40bf8022b: done: 0.009407 sec","job_status":"done","queue_duration_s":0.86271,"scheduling_latency_s":0.862723,"redis_calls":13,"redis_duration_s":0.001374,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000464,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009407,"completed_at":"2026-11-10T10:57:03.464Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444gitlab # [ 158.999237] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b544036a9fbae470023d1001","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b544036a9fbae470023d1001: start","job_status":"start","queue_duration_s":0.872985,"scheduling_latency_s":0.872999}1445gitlab # [ 159.002757] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b544036a9fbae470023d1001","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b544036a9fbae470023d1001: done: 0.009347 sec","job_status":"done","queue_duration_s":0.872985,"scheduling_latency_s":0.872999,"redis_calls":13,"redis_duration_s":0.001375,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000461,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009347,"completed_at":"2026-11-10T10:57:03.474Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 159.075474] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38359e0959cf012e720e424a","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38359e0959cf012e720e424a: start","job_status":"start","queue_duration_s":0.883672,"scheduling_latency_s":0.883685}1447gitlab # [ 159.076221] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"38359e0959cf012e720e424a","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-38359e0959cf012e720e424a: done: 0.009971 sec","job_status":"done","queue_duration_s":0.883672,"scheduling_latency_s":0.883685,"redis_calls":13,"redis_duration_s":0.001602,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009971,"completed_at":"2026-11-10T10:57:03.485Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 159.083737] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"450b2e66d4467fd75d5303b1","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-450b2e66d4467fd75d5303b1: start","job_status":"start","queue_duration_s":0.894491,"scheduling_latency_s":0.894506}1449gitlab # [ 159.123573] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.496Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"450b2e66d4467fd75d5303b1","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-450b2e66d4467fd75d5303b1: done: 0.009526 sec","job_status":"done","queue_duration_s":0.894491,"scheduling_latency_s":0.894506,"redis_calls":13,"redis_duration_s":0.001362,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00046,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009526,"completed_at":"2026-11-10T10:57:03.496Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450gitlab # [ 159.129851] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c4bc10a06c3004589d9e55d","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c4bc10a06c3004589d9e55d: start","job_status":"start","queue_duration_s":0.904827,"scheduling_latency_s":0.90484}1451gitlab # [ 159.174072] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c4bc10a06c3004589d9e55d","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c4bc10a06c3004589d9e55d: done: 0.009617 sec","job_status":"done","queue_duration_s":0.904827,"scheduling_latency_s":0.90484,"redis_calls":13,"redis_duration_s":0.0013939999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009617,"completed_at":"2026-11-10T10:57:03.506Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452gitlab # [ 159.175786] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"beb0753a9a0ea198dedbce1f","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-beb0753a9a0ea198dedbce1f: start","job_status":"start","queue_duration_s":0.915289,"scheduling_latency_s":0.915303}1453gitlab # [ 159.187118] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.517Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"beb0753a9a0ea198dedbce1f","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-beb0753a9a0ea198dedbce1f: done: 0.00954 sec","job_status":"done","queue_duration_s":0.915289,"scheduling_latency_s":0.915303,"redis_calls":13,"redis_duration_s":0.001396,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00954,"completed_at":"2026-11-10T10:57:03.517Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454gitlab # [ 159.230161] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d52241273a014250374bc4e3","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d52241273a014250374bc4e3: start","job_status":"start","queue_duration_s":0.926322,"scheduling_latency_s":0.926337}1455gitlab # [ 159.232673] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d52241273a014250374bc4e3","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d52241273a014250374bc4e3: done: 0.009804 sec","job_status":"done","queue_duration_s":0.926322,"scheduling_latency_s":0.926337,"redis_calls":13,"redis_duration_s":0.001446,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000474,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009804,"completed_at":"2026-11-10T10:57:03.528Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456gitlab # [ 159.288630] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3561f43b168aa8fce8d216e","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3561f43b168aa8fce8d216e: start","job_status":"start","queue_duration_s":0.93705,"scheduling_latency_s":0.937064}1457gitlab # [ 159.289518] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3561f43b168aa8fce8d216e","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3561f43b168aa8fce8d216e: done: 0.009419 sec","job_status":"done","queue_duration_s":0.93705,"scheduling_latency_s":0.937064,"redis_calls":13,"redis_duration_s":0.001397,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000468,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009419,"completed_at":"2026-11-10T10:57:03.538Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 159.339303] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"408f97d4ebf115e402dff018","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-408f97d4ebf115e402dff018: start","job_status":"start","queue_duration_s":0.94728,"scheduling_latency_s":0.947294}1459gitlab # [ 159.339992] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"408f97d4ebf115e402dff018","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-408f97d4ebf115e402dff018: done: 0.009429 sec","job_status":"done","queue_duration_s":0.94728,"scheduling_latency_s":0.947294,"redis_calls":13,"redis_duration_s":0.001373,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000457,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009429,"completed_at":"2026-11-10T10:57:03.548Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460gitlab # [ 159.393656] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8376a5b8729eeefa3c7c24b","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8376a5b8729eeefa3c7c24b: start","job_status":"start","queue_duration_s":0.957495,"scheduling_latency_s":0.957509}1461gitlab # [ 159.394420] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.559Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8376a5b8729eeefa3c7c24b","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8376a5b8729eeefa3c7c24b: done: 0.009562 sec","job_status":"done","queue_duration_s":0.957495,"scheduling_latency_s":0.957509,"redis_calls":13,"redis_duration_s":0.001385,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009562,"completed_at":"2026-11-10T10:57:03.559Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462gitlab # [ 159.403948] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.560Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5c2fac63b5ee5a8dcbb29c9","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5c2fac63b5ee5a8dcbb29c9: start","job_status":"start","queue_duration_s":0.967862,"scheduling_latency_s":0.967876}1463gitlab # [ 159.452666] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5c2fac63b5ee5a8dcbb29c9","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5c2fac63b5ee5a8dcbb29c9: done: 0.009518 sec","job_status":"done","queue_duration_s":0.967862,"scheduling_latency_s":0.967876,"redis_calls":13,"redis_duration_s":0.001536,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009518,"completed_at":"2026-11-10T10:57:03.569Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464gitlab # [ 159.453638] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b1bc1ba8c74352b9e247217","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b1bc1ba8c74352b9e247217: start","job_status":"start","queue_duration_s":0.978244,"scheduling_latency_s":0.978259}1465gitlab # [ 159.460638] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.580Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b1bc1ba8c74352b9e247217","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b1bc1ba8c74352b9e247217: done: 0.009606 sec","job_status":"done","queue_duration_s":0.978244,"scheduling_latency_s":0.978259,"redis_calls":13,"redis_duration_s":0.001582,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009606,"completed_at":"2026-11-10T10:57:03.580Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1466gitlab # [ 159.508714] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b426fdd299a0cac0165b8be","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b426fdd299a0cac0165b8be: start","job_status":"start","queue_duration_s":0.988934,"scheduling_latency_s":0.988948}1467gitlab # [ 159.509600] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.590Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b426fdd299a0cac0165b8be","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b426fdd299a0cac0165b8be: done: 0.009743 sec","job_status":"done","queue_duration_s":0.988934,"scheduling_latency_s":0.988948,"redis_calls":13,"redis_duration_s":0.00195,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000736,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009743,"completed_at":"2026-11-10T10:57:03.590Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468gitlab # [ 159.561299] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"141a1a279225de366781087e","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-141a1a279225de366781087e: start","job_status":"start","queue_duration_s":0.999521,"scheduling_latency_s":0.999535}1469gitlab # [ 159.561980] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"141a1a279225de366781087e","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-141a1a279225de366781087e: done: 0.009346 sec","job_status":"done","queue_duration_s":0.999521,"scheduling_latency_s":0.999535,"redis_calls":13,"redis_duration_s":0.001518,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000477,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009346,"completed_at":"2026-11-10T10:57:03.601Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1470gitlab # [ 159.606999] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b04cb9adc611c08e81f2700e","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b04cb9adc611c08e81f2700e: start","job_status":"start","queue_duration_s":1.009773,"scheduling_latency_s":1.009787}1471gitlab # [ 159.607776] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b04cb9adc611c08e81f2700e","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b04cb9adc611c08e81f2700e: done: 0.009282 sec","job_status":"done","queue_duration_s":1.009773,"scheduling_latency_s":1.009787,"redis_calls":13,"redis_duration_s":0.001447,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000463,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009282,"completed_at":"2026-11-10T10:57:03.611Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472gitlab # [ 159.668623] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2709412a5575f683f3db1ac","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2709412a5575f683f3db1ac: start","job_status":"start","queue_duration_s":1.020314,"scheduling_latency_s":1.020348}1473gitlab # [ 159.669268] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2709412a5575f683f3db1ac","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2709412a5575f683f3db1ac: done: 0.011918 sec","job_status":"done","queue_duration_s":1.020314,"scheduling_latency_s":1.020348,"redis_calls":13,"redis_duration_s":0.001559,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011918,"completed_at":"2026-11-10T10:57:03.624Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1474gitlab # [ 159.707638] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a6430f8f24278f330d386f5","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a6430f8f24278f330d386f5: start","job_status":"start","queue_duration_s":1.033205,"scheduling_latency_s":1.033219}1475gitlab # [ 159.708279] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a6430f8f24278f330d386f5","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a6430f8f24278f330d386f5: done: 0.01435 sec","job_status":"done","queue_duration_s":1.033205,"scheduling_latency_s":1.033219,"redis_calls":13,"redis_duration_s":0.005428,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000614,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004814,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01435,"completed_at":"2026-11-10T10:57:03.639Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476gitlab # [ 159.748647] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b0aff24662414a8933d32b0","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b0aff24662414a8933d32b0: start","job_status":"start","queue_duration_s":1.048594,"scheduling_latency_s":1.04861}1477gitlab # [ 159.748962] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b0aff24662414a8933d32b0","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b0aff24662414a8933d32b0: done: 0.009558 sec","job_status":"done","queue_duration_s":1.048594,"scheduling_latency_s":1.04861,"redis_calls":13,"redis_duration_s":0.00148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00048,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009558,"completed_at":"2026-11-10T10:57:03.650Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478gitlab # [ 159.796085] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.651Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27a87b7525da61e798d4e96c","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27a87b7525da61e798d4e96c: start","job_status":"start","queue_duration_s":1.058971,"scheduling_latency_s":1.058984}1479gitlab # [ 159.796815] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.661Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27a87b7525da61e798d4e96c","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27a87b7525da61e798d4e96c: done: 0.010085 sec","job_status":"done","queue_duration_s":1.058971,"scheduling_latency_s":1.058984,"redis_calls":13,"redis_duration_s":0.002049,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001499,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010085,"completed_at":"2026-11-10T10:57:03.661Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1480gitlab # [ 159.812967] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35162a3a6be1ffbc4c12d5f7","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35162a3a6be1ffbc4c12d5f7: start","job_status":"start","queue_duration_s":1.070073,"scheduling_latency_s":1.070088}1481gitlab # [ 159.851322] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.672Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35162a3a6be1ffbc4c12d5f7","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35162a3a6be1ffbc4c12d5f7: done: 0.01025 sec","job_status":"done","queue_duration_s":1.070073,"scheduling_latency_s":1.070088,"redis_calls":13,"redis_duration_s":0.001748,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000714,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006518,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01025,"completed_at":"2026-11-10T10:57:03.672Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482gitlab # [ 159.890814] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"841a9d266b29a262b8f6d13c","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-841a9d266b29a262b8f6d13c: start","job_status":"start","queue_duration_s":1.081189,"scheduling_latency_s":1.081203}1483gitlab # [ 159.891579] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.683Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"841a9d266b29a262b8f6d13c","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-841a9d266b29a262b8f6d13c: done: 0.009683 sec","job_status":"done","queue_duration_s":1.081189,"scheduling_latency_s":1.081203,"redis_calls":13,"redis_duration_s":0.001693,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00114,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009683,"completed_at":"2026-11-10T10:57:03.683Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484gitlab # [ 159.903750] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05b46cb7c0c1ffec20d9d639","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05b46cb7c0c1ffec20d9d639: start","job_status":"start","queue_duration_s":1.091736,"scheduling_latency_s":1.09175}1485gitlab # [ 159.943716] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05b46cb7c0c1ffec20d9d639","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05b46cb7c0c1ffec20d9d639: done: 0.009571 sec","job_status":"done","queue_duration_s":1.091736,"scheduling_latency_s":1.09175,"redis_calls":13,"redis_duration_s":0.001616,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009571,"completed_at":"2026-11-10T10:57:03.693Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1486gitlab # [ 159.949499] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd41e8064a56da18f330658f","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd41e8064a56da18f330658f: start","job_status":"start","queue_duration_s":1.102336,"scheduling_latency_s":1.10235}1487gitlab # [ 159.989807] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd41e8064a56da18f330658f","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd41e8064a56da18f330658f: done: 0.009377 sec","job_status":"done","queue_duration_s":1.102336,"scheduling_latency_s":1.10235,"redis_calls":13,"redis_duration_s":0.0015019999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000486,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009377,"completed_at":"2026-11-10T10:57:03.703Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488gitlab # [ 160.029652] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aba3164fe1daccd78c910e51","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aba3164fe1daccd78c910e51: start","job_status":"start","queue_duration_s":1.112613,"scheduling_latency_s":1.112626}1489gitlab # [ 160.030426] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aba3164fe1daccd78c910e51","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aba3164fe1daccd78c910e51: done: 0.009267 sec","job_status":"done","queue_duration_s":1.112613,"scheduling_latency_s":1.112626,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000472,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009267,"completed_at":"2026-11-10T10:57:03.714Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1490gitlab # [ 160.070456] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.715Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bc2ab8cabad89b582d87d82","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bc2ab8cabad89b582d87d82: start","job_status":"start","queue_duration_s":1.122699,"scheduling_latency_s":1.122713}1491gitlab # [ 160.071151] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bc2ab8cabad89b582d87d82","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bc2ab8cabad89b582d87d82: done: 0.009124 sec","job_status":"done","queue_duration_s":1.122699,"scheduling_latency_s":1.122713,"redis_calls":13,"redis_duration_s":0.001472,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009124,"completed_at":"2026-11-10T10:57:03.724Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1492gitlab # [ 160.117786] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de66d224918d4c61aa166d6d","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de66d224918d4c61aa166d6d: start","job_status":"start","queue_duration_s":1.132722,"scheduling_latency_s":1.132735}1493gitlab # [ 160.118687] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.734Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de66d224918d4c61aa166d6d","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de66d224918d4c61aa166d6d: done: 0.009389 sec","job_status":"done","queue_duration_s":1.132722,"scheduling_latency_s":1.132735,"redis_calls":13,"redis_duration_s":0.001565,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009389,"completed_at":"2026-11-10T10:57:03.734Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1494gitlab # [ 160.132683] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95bea1cd68a62da90b4f0f6a","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95bea1cd68a62da90b4f0f6a: start","job_status":"start","queue_duration_s":1.143186,"scheduling_latency_s":1.1432}1495gitlab # [ 160.144704] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95bea1cd68a62da90b4f0f6a","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95bea1cd68a62da90b4f0f6a: done: 0.009284 sec","job_status":"done","queue_duration_s":1.143186,"scheduling_latency_s":1.1432,"redis_calls":13,"redis_duration_s":0.001455,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000467,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009284,"completed_at":"2026-11-10T10:57:03.744Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1496gitlab # [ 160.195987] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e7a4e35a925503755514965","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e7a4e35a925503755514965: start","job_status":"start","queue_duration_s":1.153288,"scheduling_latency_s":1.153301}1497gitlab # [ 160.196772] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e7a4e35a925503755514965","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e7a4e35a925503755514965: done: 0.009411 sec","job_status":"done","queue_duration_s":1.153288,"scheduling_latency_s":1.153301,"redis_calls":13,"redis_duration_s":0.001516,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000474,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009411,"completed_at":"2026-11-10T10:57:03.754Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1498gitlab # [ 160.239084] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63e07ca85fca8b7bf43dd5df","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63e07ca85fca8b7bf43dd5df: start","job_status":"start","queue_duration_s":1.163519,"scheduling_latency_s":1.163532}1499gitlab # [ 160.239772] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63e07ca85fca8b7bf43dd5df","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63e07ca85fca8b7bf43dd5df: done: 0.009385 sec","job_status":"done","queue_duration_s":1.163519,"scheduling_latency_s":1.163532,"redis_calls":13,"redis_duration_s":0.001516,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009385,"completed_at":"2026-11-10T10:57:03.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500gitlab # [ 160.280109] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d8b138ce4db0dc4ce760b5d","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d8b138ce4db0dc4ce760b5d: start","job_status":"start","queue_duration_s":1.174099,"scheduling_latency_s":1.174112}1501gitlab # [ 160.280982] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:03.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d8b138ce4db0dc4ce760b5d","created_at":"2026-11-10T10:57:02.544Z","trace_propagation_headers":{"sentry-trace":"c22e227dffd74b4584a651d73d8490cf-32cfcd807c0d47c0","baggage":"sentry-trace_id=c22e227dffd74b4584a651d73d8490cf,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"38b46dbc3dd8f04f585c6b21d2097708","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:02.592Z","job_size_bytes":2,"pid":1622,"sidekiq_tid":"made","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d8b138ce4db0dc4ce760b5d: done: 0.009441 sec","job_status":"done","queue_duration_s":1.174099,"scheduling_latency_s":1.174112,"redis_calls":13,"redis_duration_s":0.001611,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009441,"completed_at":"2026-11-10T10:57:03.775Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1502gitlab # [ 160.504419] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2079805001 wd_nsec: 20798038181503gitlab # [ 163.508415] setup-gitlab-start[2227]: W, [2026-11-10T10:57:08.934401 #2227] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1504gitlab # [ 185.411716] postgres[2442]: [2442] ERROR: relation "table_without_models" does not exist1505gitlab # [ 185.412676] postgres[2442]: [2442] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1506gitlab # [ 188.328374] runuser[2226]: pam_unix(runuser:session): session closed for user gitlab1507gitlab # [ 188.329532] setup-gitlab-start[2201]: + systemctl restart gitlab-sidekiq.service1508gitlab # [ 188.348452] systemd[1]: Stopping gitlab-sidekiq.service...1509gitlab # [ 188.349663] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:33.775Z","message":"Shutting down"}1510gitlab # [ 188.351068] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:33.775Z","message":"Scheduler exiting..."}1511gitlab # [ 188.352439] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:33.776Z","message":"Terminating quiet threads for default capsule"}1512gitlab # [ 188.354170] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:33.776Z","message":"Scheduler exiting..."}1513gitlab # [ 188.852728] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:34.279Z","message":"Bye!"}1514gitlab # [ 188.876508] sidekiq-cluster[1622]: {"severity":"INFO","time":"2026-11-10T10:57:34.303Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1622,"worker_id":"sidekiq_0","memwd_rss_bytes":941957120,"retry":0}1515gitlab # [ 192.061329] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1516gitlab # [ 192.062245] systemd[1]: Stopped gitlab-sidekiq.service.1517gitlab # [ 192.063048] systemd[1]: gitlab-sidekiq.service: Consumed 58.867s CPU time over 1min 58.365s wall clock time, 958M memory peak, 4.4M read from disk.1518gitlab # [ 192.067059] systemd[1]: Started gitlab-sidekiq.service.1519gitlab # [ 192.074210] setup-gitlab-start[2201]: + mkdir -p /var/lib/secrets1520gitlab # [ 192.077147] setup-gitlab-start[2201]: + echo nixbot-test-token-of-20-chars1521gitlab # [ 192.079717] setup-gitlab-start[2201]: + chmod 644 /var/lib/secrets/gitlab-token1522gitlab # [ 192.082218] setup-gitlab-start[2201]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1523gitlab # [ 192.540076] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:37 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 4421524gitlab # [ 193.098696] gitaly[910]: time="2026-11-10T10:57:38.524Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T10:57:48.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:38.524" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1525gitlab # [ 193.098962] gitaly[910]: time="2026-11-10T10:57:38.525Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T10:57:48.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:38.524" grpc.time_ms=0.482 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1526gitlab # [ 193.246888] gitaly[910]: time="2026-11-10T10:57:38.673Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T10:57:48.152" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:38.673" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1527gitlab # [ 193.247189] gitaly[910]: time="2026-11-10T10:57:38.673Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T10:57:48.152" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:38.673" grpc.time_ms=0.411 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1528gitlab # [ 193.352699] gitaly[910]: time="2026-11-10T10:57:38.779Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ diskcache=da8bf990-e9bf-4e83-b7f0-0953acc71390 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T10:57:48.206" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:38.726" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1529gitlab # [ 193.353160] gitaly[910]: time="2026-11-10T10:57:38.779Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1577 command.oublock=32 command.real_time_ms=23 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T10:57:48.206" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:38.726" grpc.time_ms=52.693 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1530gitlab # [ 193.401085] gitaly[910]: time="2026-11-10T10:57:38.827Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T10:57:48.306" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:38.826" pid=910 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1531gitlab # [ 193.401828] gitaly[910]: time="2026-11-10T10:57:38.828Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T10:57:48.306" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:38.826" grpc.time_ms=0.842 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1532gitlab # [ 193.420245] gitaly[910]: time="2026-11-10T10:57:38.846Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T10:57:48.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:38.834" grpc.time_ms=12.022 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1533gitlab # [ 193.507835] gitaly[910]: time="2026-11-10T10:57:38.934Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T10:57:48.103" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:38.934" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1534gitlab # [ 193.508169] gitaly[910]: time="2026-11-10T10:57:38.934Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T10:57:48.103" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:38.934" grpc.time_ms=0.395 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1535gitlab # [ 193.552127] gitaly[910]: time="2026-11-10T10:57:38.978Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ diskcache=a0b0ac41-4cb6-4434-a057-82b965268af7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T10:57:48.107" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:38.938" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1536gitlab # [ 193.553187] gitaly[910]: time="2026-11-10T10:57:38.978Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1570 command.oublock=32 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-11-10T10:57:48.107" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:38.938" grpc.time_ms=40.759 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1537gitlab # [ 193.577823] gitaly[910]: time="2026-11-10T10:57:39.004Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T10:57:48.173" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:39.004" pid=910 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1538gitlab # [ 193.578771] gitaly[910]: time="2026-11-10T10:57:39.004Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T10:57:48.173" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:39.004" grpc.time_ms=0.403 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1539gitlab # [ 193.609657] gitaly[910]: time="2026-11-10T10:57:39.035Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T10:57:49.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:39.027" grpc.time_ms=8.601 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1540gitlab # [ 193.657438] gitaly[910]: time="2026-11-10T10:57:39.084Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T10:57:49.063" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:39.083" grpc.time_ms=0.323 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1541gitlab # [ 193.668783] gitaly[910]: time="2026-11-10T10:57:39.095Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T10:57:49.074" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:39.094" grpc.time_ms=0.37 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1542gitlab # [ 194.132655] gitaly[910]: time="2026-11-10T10:57:39.559Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPE65R7FES3XZ44646ZPQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T10:57:49.027" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:39.548" grpc.time_ms=10.525 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1543gitlab # [ 194.397146] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:39 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 18321544gitlab # [ 194.398688] setup-gitlab-start[2498]: {"id":1,"message":null,"error":null}1545gitlab # [ 195.460993] sidekiq-cluster[2481]: {"severity":"INFO","time":"2026-11-10T10:57:40.868Z","message":"Starting cluster with 1 processes"}1546gitlab # [ 199.536375] gitaly[910]: time="2026-11-10T10:57:44.962Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPMWMM4R7V6BT3TGB8CB2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T10:57:54.121" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:44.951" grpc.time_ms=10.766 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1547gitlab # [ 199.612704] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:45 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 1831548gitlab # [ 199.618882] setup-gitlab-start[2201]: + mkdir -p /tmp/test-flake1549gitlab # [ 199.623107] setup-gitlab-start[2201]: + cd /tmp/test-flake1550gitlab # [ 199.623707] setup-gitlab-start[2201]: + git init -b master1551gitlab # [ 199.644891] setup-gitlab-start[2572]: Initialized empty Git repository in /tmp/test-flake/.git/1552gitlab # [ 199.645662] setup-gitlab-start[2201]: + git config user.name test1553gitlab # [ 199.651555] setup-gitlab-start[2201]: + git config user.email test@example.com1554gitlab # [ 199.655824] setup-gitlab-start[2201]: + cat1555gitlab # [ 199.660951] setup-gitlab-start[2201]: + git add flake.nix1556gitlab # [ 199.667046] setup-gitlab-start[2201]: + git commit -m 'initial commit'1557gitlab # [ 199.682063] setup-gitlab-start[2577]: [master (root-commit) b05bdda] initial commit1558gitlab # [ 199.682193] setup-gitlab-start[2577]: 1 file changed, 10 insertions(+)1559gitlab # [ 199.682944] setup-gitlab-start[2577]: create mode 100644 flake.nix1560gitlab # [ 199.688172] setup-gitlab-start[2201]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1561gitlab # [ 199.698566] setup-gitlab-start[2201]: + git push -u origin master1562gitlab # [ 199.840795] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:45 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 1081563gitlab # [ 200.041601] gitaly[910]: time="2026-11-10T10:57:45.468Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HPN9RGEQVGKKX0TH48S0Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:45.458" grpc.time_ms=9.8 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1564gitlab # [ 200.071871] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:45 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 2251565gitlab # [ 200.331747] gitaly[910]: time="2026-11-10T10:57:45.758Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T10:57:55.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:45.747" grpc.time_ms=10.955 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1566gitlab # [ 200.393083] gitaly[910]: time="2026-11-10T10:57:45.819Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-10T10:58:15.012" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-10T10:57:45.813" grpc.time_ms=6.209 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1567gitlab # [ 200.435404] gitaly[910]: time="2026-11-10T10:57:45.862Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M72HPNHC58J4F129DPYHBM88 duration_ms=180ns method=POST pid=910 status=200 url="http://unix/api/v4/internal/allowed"1568gitlab # [ 200.460068] gitaly[910]: time="2026-11-10T10:57:45.886Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M72HPNHC58J4F129DPYHBM88 duration_ms=23ns method=POST pid=910 status=200 url="http://unix/api/v4/internal/pre_receive"1569gitlab # [ 200.460617] gitaly[910]: time="2026-11-10T10:57:45.887Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:57:45.681" grpc.time_ms=205.718 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1570gitlab # [ 200.517879] gitaly[910]: time="2026-11-10T10:57:45.944Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:57:45.944" grpc.time_ms=0.469 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1571gitlab # [ 200.570483] gitaly[910]: time="2026-11-10T10:57:45.996Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:57:45.996" grpc.time_ms=0.349 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1572gitlab # [ 200.608032] gitaly[910]: time="2026-11-10T10:57:46.034Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:57:46.033" grpc.time_ms=0.409 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1573gitlab # [ 200.719890] gitaly[910]: time="2026-11-10T10:57:46.146Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T10:57:56.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:46.138" grpc.time_ms=7.955 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1574gitlab # [ 200.742702] gitaly[910]: time="2026-11-10T10:57:46.169Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1557 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-10T10:57:56.021" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:57:46.152" grpc.time_ms=16.446 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1575gitlab # [ 200.781312] gitaly[910]: time="2026-11-10T10:57:46.208Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M72HPNHC58J4F129DPYHBM88 duration_ms=145ns method=POST pid=910 status=200 url="http://unix/api/v4/internal/post_receive"1576gitlab # [ 200.782225] gitaly[910]: time="2026-11-10T10:57:46.208Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:57:46.061" grpc.time_ms=147.153 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1577gitlab # [ 200.803370] gitaly[910]: time="2026-11-10T10:57:46.230Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 diskcache=6d3b48ce-e016-45a6-a943-9e5489bf5d21 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:45.612" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1578gitlab # [ 200.804128] gitaly[910]: time="2026-11-10T10:57:46.230Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=199 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13688 command.oublock=32 command.real_time_ms=607 command.spawn_token_fork_ms=0 command.system_time_ms=138 command.user_time_ms=61 component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:45.612" grpc.time_ms=617.73 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1579gitlab # [ 200.805846] gitlab-workhorse[823]: time="2026-11-10T10:57:46Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521580gitlab # [ 200.807263] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:46 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 7141581gitlab # [ 200.809064] setup-gitlab-start[2581]: To http://gitlab/root/test-flake.git1582gitlab # [ 200.809855] setup-gitlab-start[2581]: * [new branch] master -> master1583gitlab # [ 200.830599] setup-gitlab-start[2581]: branch 'master' set up to track 'origin/master'.1584gitlab # [ 200.833719] systemd[1]: Finished setup-gitlab.service.1585gitlab # [ 200.835899] systemd[1]: Starting nixbot CI service...1586gitlab: (finished: waiting for unit setup-gitlab.service, in 202.08 seconds)1587gitlab: waiting for unit nixbot.service1588gitlab # [ 202.070241] nixbot[2667]: {"ts": "2026-11-10T10:57:47.496Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1589gitlab # [ 202.103446] nixbot[2667]: {"ts": "2026-11-10T10:57:47.530Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1590gitlab # [ 202.135713] nixbot[2667]: {"ts": "2026-11-10T10:57:47.562Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1591gitlab # [ 202.141943] nixbot[2667]: {"ts": "2026-11-10T10:57:47.568Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1592gitlab # [ 202.151844] nixbot[2667]: {"ts": "2026-11-10T10:57:47.578Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1593gitlab # [ 202.164090] nixbot[2667]: {"ts": "2026-11-10T10:57:47.590Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1594gitlab # [ 202.173809] nixbot[2667]: {"ts": "2026-11-10T10:57:47.600Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1595gitlab # [ 202.181537] nixbot[2667]: {"ts": "2026-11-10T10:57:47.608Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1596gitlab # [ 202.190592] nixbot[2667]: {"ts": "2026-11-10T10:57:47.617Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1597gitlab # [ 202.198207] nixbot[2667]: {"ts": "2026-11-10T10:57:47.624Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1598gitlab # [ 202.205839] nixbot[2667]: {"ts": "2026-11-10T10:57:47.632Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1599gitlab # [ 202.219435] nixbot[2667]: {"ts": "2026-11-10T10:57:47.645Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1600gitlab # [ 202.228988] nixbot[2667]: {"ts": "2026-11-10T10:57:47.655Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1601gitlab # [ 202.237098] nixbot[2667]: {"ts": "2026-11-10T10:57:47.663Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1602gitlab # [ 202.242996] nixbot[2667]: {"ts": "2026-11-10T10:57:47.669Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1603gitlab # [ 202.251363] nixbot[2667]: {"ts": "2026-11-10T10:57:47.677Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1604gitlab # [ 202.254826] nixbot[2667]: {"ts": "2026-11-10T10:57:47.681Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1605gitlab # [ 202.263308] nixbot[2667]: {"ts": "2026-11-10T10:57:47.689Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1606gitlab # [ 202.274844] nixbot[2667]: {"ts": "2026-11-10T10:57:47.701Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1607gitlab # [ 202.282272] nixbot[2667]: {"ts": "2026-11-10T10:57:47.708Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1608gitlab # [ 202.289742] nixbot[2667]: {"ts": "2026-11-10T10:57:47.716Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1609gitlab # [ 202.300623] nixbot[2667]: {"ts": "2026-11-10T10:57:47.727Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1610gitlab # [ 202.303880] nixbot[2667]: {"ts": "2026-11-10T10:57:47.730Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1611gitlab # [ 202.307501] nixbot[2667]: {"ts": "2026-11-10T10:57:47.734Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1612gitlab # [ 202.310191] nixbot[2667]: {"ts": "2026-11-10T10:57:47.736Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1613gitlab # [ 202.314288] nixbot[2667]: {"ts": "2026-11-10T10:57:47.740Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1614gitlab # [ 202.317520] nixbot[2667]: {"ts": "2026-11-10T10:57:47.744Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1615gitlab # [ 202.320544] nixbot[2667]: {"ts": "2026-11-10T10:57:47.747Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1616gitlab # [ 202.323757] nixbot[2667]: {"ts": "2026-11-10T10:57:47.750Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1617gitlab # [ 202.340137] nixbot[2667]: {"ts": "2026-11-10T10:57:47.766Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1618gitlab # [ 202.349233] nixbot[2667]: {"ts": "2026-11-10T10:57:47.775Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1619gitlab # [ 202.356461] nixbot[2667]: {"ts": "2026-11-10T10:57:47.782Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1620gitlab # [ 202.359574] nixbot[2667]: {"ts": "2026-11-10T10:57:47.786Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1621gitlab # [ 202.370574] nixbot[2667]: {"ts": "2026-11-10T10:57:47.797Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1622gitlab # [ 202.373649] nixbot[2667]: {"ts": "2026-11-10T10:57:47.800Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1623gitlab # [ 202.381997] nixbot[2667]: {"ts": "2026-11-10T10:57:47.808Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1624gitlab # [ 202.390263] nixbot[2667]: {"ts": "2026-11-10T10:57:47.816Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1625gitlab # [ 202.399775] nixbot[2667]: {"ts": "2026-11-10T10:57:47.826Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "effect_when_changed"}1626gitlab # [ 202.408134] nixbot[2667]: {"ts": "2026-11-10T10:57:47.834Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 38, "migration": "commit_statuses"}1627gitlab # [ 202.756752] nixbot[2667]: {"ts": "2026-11-10T10:57:48.183Z", "level": "info", "logger": "uvicorn.error", "msg": "Started server process [2667]", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"}1628gitlab # [ 202.757109] nixbot[2667]: {"ts": "2026-11-10T10:57:48.183Z", "level": "info", "logger": "uvicorn.error", "msg": "Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)"}1629gitlab # [ 202.777035] nixbot[2667]: {"ts": "2026-11-10T10:57:48.203Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:55498 - \"GET /health HTTP/1.1\" 200"}1630gitlab # [ 202.786786] systemd[1]: Started nixbot CI service.1631gitlab # [ 202.788677] systemd[1]: Reached target Multi-User System.1632gitlab # [ 202.791278] systemd[1]: Startup finished in 1.093s (kernel) + 3.492s (initrd) + 3min 18.204s (userspace) = 3min 22.791s.1633gitlab # [ 203.492229] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:48 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 7151634gitlab # [ 203.510343] nixbot[2667]: {"ts": "2026-11-10T10:57:48.936Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1635gitlab: (finished: waiting for unit nixbot.service, in 2.09 seconds)1636gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1637gitlab # [ 203.584304] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 561638gitlab # [ 203.586766] nixbot[2667]: {"ts": "2026-11-10T10:57:49.013Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1639gitlab # [ 203.590322] nixbot[2667]: {"ts": "2026-11-10T10:57:49.016Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:50048 - \"GET /health HTTP/1.1\" 200"}1640gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1641subtest: project discovered and webhook registered1642gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1643gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.07 seconds)1644gitlab # [ 203.667356] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 481645gitlab # [ 203.686865] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:49 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 951646gitlab # [ 203.846798] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:49 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1561647gitlab # [ 203.919255] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 631648gitlab # [ 203.969349] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:49 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 451649gitlab # [ 204.023122] gitaly[910]: time="2026-11-10T10:57:49.449Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPSASQVHFV8HAAYDDAME1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-11-10T10:58:19.035" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:57:49.435" grpc.time_ms=13.091 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1650gitlab # [ 204.056988] gitaly[910]: time="2026-11-10T10:57:49.483Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPSASQVHFV8HAAYDDAME1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:58:19.082" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:57:49.482" grpc.time_ms=0.72 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1651gitlab # [ 204.083033] gitaly[910]: time="2026-11-10T10:57:49.509Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPSASQVHFV8HAAYDDAME1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-11-10T10:57:59.023" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:57:49.503" grpc.time_ms=6.008 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1652gitlab # [ 204.105670] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1301653gitlab # [ 204.242286] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1331654gitlab # [ 204.251986] nixbot[2667]: {"ts": "2026-11-10T10:57:49.678Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1655gitlab # [ 204.376466] gitaly[910]: time="2026-11-10T10:57:49.803Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HPSNRM6YDGFW3JNAZTSFC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:49.795" grpc.time_ms=7.803 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1656gitlab # [ 204.376779] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:49 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 511657gitlab # [ 204.481094] gitaly[910]: time="2026-11-10T10:57:49.907Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPSRDF1QRZMVJ426TTM0A grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:49.897" pid=910 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1658gitlab # [ 204.503137] gitaly[910]: time="2026-11-10T10:57:49.929Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPSRDF1QRZMVJ426TTM0A grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:49.897" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1659gitlab # [ 204.528966] gitaly[910]: time="2026-11-10T10:57:49.955Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPSRDF1QRZMVJ426TTM0A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:49.897" grpc.time_ms=57.655 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1660gitlab # [ 204.530272] gitlab-workhorse[823]: time="2026-11-10T10:57:49Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131661gitlab # [ 204.531266] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:49 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1191662gitlab # [ 204.588259] gitaly[910]: time="2026-11-10T10:57:50.014Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPSW9G5EQG4BJGV70FYMH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:50.002" pid=910 remote_ip="2001:db8:1::1" request_sha=407b89b11be23a1f6dc6e4c57e1084a14691434d response_bytes=223 span.kind=server system=grpc1663gitlab # [ 204.600950] gitaly[910]: time="2026-11-10T10:57:50.015Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1199 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPSW9G5EQG4BJGV70FYMH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:50.002" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1664gitlab # [ 204.613938] gitaly[910]: time="2026-11-10T10:57:50.040Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1199 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPSW9G5EQG4BJGV70FYMH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:50.002" grpc.time_ms=25.725 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1665gitlab # [ 204.628403] gitlab-workhorse[823]: time="2026-11-10T10:57:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231666gitlab # [ 204.628934] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 791667gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1668gitlab # [ 204.689835] gitaly[910]: time="2026-11-10T10:57:50.116Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M72HPSZR8A3HDQ51FCNTX1MA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:50.109" grpc.time_ms=6.518 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1669gitlab # [ 204.693059] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:50 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 461670gitlab # [ 204.747674] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:50 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 551671gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.09 seconds)1672(finished: subtest: project discovered and webhook registered, in 1.16 seconds)1673subtest: push delivers the webhook1674gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1675gitlab # [ 204.763845] gitaly[910]: time="2026-11-10T10:57:50.190Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPT1E59NFRTKP6GYC127Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:50.182" pid=910 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1676gitlab # [ 204.764135] gitaly[910]: time="2026-11-10T10:57:50.190Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPT1E59NFRTKP6GYC127Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:50.182" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1677gitlab # [ 204.764683] gitaly[910]: time="2026-11-10T10:57:50.190Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPT1E59NFRTKP6GYC127Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:50.182" grpc.time_ms=7.913 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1678gitlab # [ 204.765940] gitlab-workhorse[823]: time="2026-11-10T10:57:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671679gitlab # [ 204.767662] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 641680gitlab # [ 204.873365] gitaly[910]: time="2026-11-10T10:57:50.299Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HPT5KESV9T0RQ0N5ZKVA2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:50.293" grpc.time_ms=6.164 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1681gitlab # [ 204.885779] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:50 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 531682gitlab # [ 204.934861] gitaly[910]: time="2026-11-10T10:57:50.361Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPT79DA0GB6MGHRQSC1JF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:50.350" pid=910 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1683gitlab # [ 204.935353] gitaly[910]: time="2026-11-10T10:57:50.361Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1172 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPT79DA0GB6MGHRQSC1JF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:50.350" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1684gitlab # [ 204.956477] gitaly[910]: time="2026-11-10T10:57:50.383Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1172 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPT79DA0GB6MGHRQSC1JF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:57:50.350" grpc.time_ms=32.736 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1685gitlab # [ 204.969852] gitlab-workhorse[823]: time="2026-11-10T10:57:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091686gitlab # [ 204.970516] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 821687gitlab # [ 205.047558] gitaly[910]: time="2026-11-10T10:57:50.473Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:58:20.072" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:57:50.473" grpc.time_ms=0.563 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1688gitlab # [ 205.100833] gitaly[910]: time="2026-11-10T10:57:50.527Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-10T10:58:20.121" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:57:50.521" grpc.time_ms=5.853 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1689gitlab # [ 205.141666] gitaly[910]: time="2026-11-10T10:57:50.568Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:00.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:57:50.559" grpc.time_ms=8.87 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1690gitlab # [ 205.169734] gitaly[910]: time="2026-11-10T10:57:50.596Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:00.038" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:57:50.588" grpc.time_ms=7.999 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1691gitlab # [ 205.247743] gitaly[910]: time="2026-11-10T10:57:50.674Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T10:58:00.123" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:50.673" grpc.time_ms=0.438 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1692gitlab # [ 205.289557] gitaly[910]: time="2026-11-10T10:57:50.716Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T10:58:00.154" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:57:50.705" grpc.time_ms=10.951 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1693gitlab # [ 205.379371] gitaly[910]: time="2026-11-10T10:57:50.806Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:57:50.804" grpc.time_ms=1.12 pid=910 span.kind=server system=grpc1694gitlab # [ 205.437127] gitaly[910]: time="2026-11-10T10:57:50.863Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:57:50.863" grpc.time_ms=0.486 pid=910 span.kind=server system=grpc1695gitlab # [ 205.462436] gitaly[910]: time="2026-11-10T10:57:50.889Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 diskcache=b9e93a34-97f8-401d-bb88-8ccbea7988d7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T10:58:00.004" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:50.745" pid=910 ref=refs/keep-around/b05bdda260e45ffb6084becbb5b4f5bb934cf14f remote_ip="2001:db8:1::1" revision=b05bdda260e45ffb6084becbb5b4f5bb934cf14f span.kind=server system=grpc username=root1696gitlab # [ 205.466179] gitaly[910]: time="2026-11-10T10:57:50.892Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:58:20.091" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:57:50.892" grpc.time_ms=0.376 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1697gitlab # [ 205.484210] gitaly[910]: time="2026-11-10T10:57:50.910Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=132 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5489 command.oublock=8 command.real_time_ms=135 command.spawn_token_fork_ms=0 command.system_time_ms=89 command.user_time_ms=43 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T10:58:00.004" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:57:50.745" grpc.time_ms=144.076 pid=910 ref=refs/keep-around/b05bdda260e45ffb6084becbb5b4f5bb934cf14f remote_ip="2001:db8:1::1" revision=b05bdda260e45ffb6084becbb5b4f5bb934cf14f span.kind=server system=grpc username=root1698gitlab # [ 205.608872] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:57:51 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b05bdda260e45ffb6084becbb5b4f5bb934cf14f HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 5991699gitlab # [ 205.629892] nixbot[2667]: {"ts": "2026-11-10T10:57:51.056Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-b05bdda260e4-0"}1700gitlab # [ 205.811766] nixbot[2667]: {"ts": "2026-11-10T10:57:51.238Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-b05bdda260e4-0", "argv0": "bwrap"}1701gitlab # [ 205.894621] systemd[1]: Started Nix Daemon.1702gitlab # [ 205.955366] nix-daemon[2832]: accepted connection from pid 2827, user nixbot1703gitlab # [ 210.959857] nix-daemon[2832]: accepted connection from pid 2875, user nixbot1704gitlab # [ 210.971111] nix-daemon[2832]: accepted connection from pid 2827, user nixbot1705gitlab # [ 215.600699] nix-daemon[2832]: accepted connection from pid 2827, user nixbot1706gitlab # [ 215.694482] nix-daemon[2832]: accepted connection from pid 2924, user nixbot1707gitlab # [ 215.728676] gitaly[910]: time="2026-11-10T10:58:01.153Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HQ4R7MYHM4R2EW82DRBRV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:58:31.052" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:01.153" grpc.time_ms=0.442 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1708gitlab # [ 215.820337] gitaly[910]: time="2026-11-10T10:58:01.246Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72HQ4R7MYHM4R2EW82DRBRV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:11.216" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:01.236" grpc.time_ms=9.826 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1709gitlab # [ 215.901255] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:01 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b05bdda260e45ffb6084becbb5b4f5bb934cf14f HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2311710gitlab # [ 215.947704] gitaly[910]: time="2026-11-10T10:58:01.374Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HQ4ZW974DY84P6JEER49M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:58:31.273" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:01.373" grpc.time_ms=0.572 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1711gitlab # [ 215.985200] gitaly[910]: time="2026-11-10T10:58:01.411Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HQ4ZW974DY84P6JEER49M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:11.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:01.406" grpc.time_ms=5.264 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1712gitlab # [ 216.019263] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:01 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b05bdda260e45ffb6084becbb5b4f5bb934cf14f HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1051713gitlab # [ 220.237623] nix-daemon[2832]: accepted connection from pid 2992, user nixbot1714gitlab # [ 220.289494] gitaly[910]: time="2026-11-10T10:58:05.715Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HQ97GJTDC862EF6EFZ22W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:58:35.314" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:05.715" grpc.time_ms=0.474 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1715gitlab # [ 220.310174] gitaly[910]: time="2026-11-10T10:58:05.736Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M72HQ97GJTDC862EF6EFZ22W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:15.181" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:05.731" grpc.time_ms=5.227 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1716gitlab # [ 220.346421] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:05 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b05bdda260e45ffb6084becbb5b4f5bb934cf14f HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 921717gitlab # [ 220.531369] nixbot[2667]: {"ts": "2026-11-10T10:58:05.957Z", "level": "info", "logger": "nixbot.effects_run", "msg": "effects discovered", "build_id": 1, "sha": "b05bdda260e45ffb6084becbb5b4f5bb934cf14f", "branch": "master", "pr": null, "allowed": true, "effects_started": false, "effects": []}1718gitlab # [ 220.616639] gitaly[910]: time="2026-11-10T10:58:06.043Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HQ9HEFSAK0J8HA3FMAB06 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:06.033" grpc.time_ms=9.633 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1719gitlab # [ 220.644261] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:06 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 451720gitlab # [ 220.694154] gitaly[910]: time="2026-11-10T10:58:06.120Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72HQ9JXS9ZV6T2SQCM0WYR9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:06.110" pid=910 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1721gitlab # [ 220.716361] gitaly[910]: time="2026-11-10T10:58:06.120Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HQ9JXS9ZV6T2SQCM0WYR9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:06.110" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1722gitlab # [ 220.736452] gitaly[910]: time="2026-11-10T10:58:06.121Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HQ9JXS9ZV6T2SQCM0WYR9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:06.110" grpc.time_ms=10.581 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1723gitlab # [ 220.749681] gitlab-workhorse[823]: time="2026-11-10T10:58:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671724gitlab # [ 220.751675] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1301725gitlab # [ 220.811066] gitaly[910]: time="2026-11-10T10:58:06.237Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HQ9QNDSEHDZDQQYR7GE2J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:06.230" grpc.time_ms=7.24 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1726gitlab # [ 220.811906] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:06 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 401727gitlab # [ 220.857212] gitaly[910]: time="2026-11-10T10:58:06.283Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72HQ9S08BVTDDPNMRPVRR9H grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:06.277" pid=910 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1728gitlab # [ 220.857604] gitaly[910]: time="2026-11-10T10:58:06.284Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72HQ9S08BVTDDPNMRPVRR9H grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:06.277" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1729gitlab # [ 220.878624] gitaly[910]: time="2026-11-10T10:58:06.305Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72HQ9S08BVTDDPNMRPVRR9H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:06.277" grpc.time_ms=27.778 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1730gitlab # [ 220.891771] gitlab-workhorse[823]: time="2026-11-10T10:58:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671731gitlab # [ 220.893751] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 771732gitlab # [ 221.621269] sidekiq-cluster[2545]: 2026-11-10T10:58:07.047Z pid=2545 tid=xuh INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2545, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>430358528}1733gitlab # [ 221.806841] sidekiq-cluster[2545]: W, [2026-11-10T10:58:07.233417 #2545] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1734gitlab # [ 221.845704] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:07.272Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1735gitlab # [ 223.529080] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:08.954Z","message":"GitLab reliable fetch activated!"}1736gitlab # [ 223.529797] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:08.955Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1737gitlab # [ 244.250165] postgres[3228]: [3228] ERROR: relation "table_without_models" does not exist1738gitlab # [ 244.250897] postgres[3228]: [3228] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1739gitlab # [ 246.697759] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:32.120Z","message":"Booted Rails 7.2.3.1 application in production environment"}1740gitlab # [ 246.698567] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:32.120Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"}1741gitlab # [ 246.699707] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:32.120Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1742gitlab # [ 246.703689] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:32.121Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1743gitlab # [ 246.705309] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:32.123Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1744gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 42.41 seconds)1745gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1746gitlab # [ 247.261146] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:32 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 421747gitlab # [ 247.351429] gitaly[910]: time="2026-11-10T10:58:32.777Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR3KNKZ4KVP889W0Y422P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:32.768" grpc.time_ms=8.472 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1748gitlab # [ 247.382190] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:32 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 1151749gitlab # Enumerating objects: 5, done.1750gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1751gitlab # Delta compression using up to 4 threads1752gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1753gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 272 bytes | 136.00 KiB/s, done.1754gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1755gitlab # [ 247.595295] gitaly[910]: time="2026-11-10T10:58:33.021Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=328 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-11-10T10:59:03.010" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-11-10T10:58:33.010" grpc.time_ms=11.518 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1756gitlab # [ 247.606790] gitaly[910]: time="2026-11-10T10:58:33.033Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF duration_ms=69ns method=POST pid=910 status=200 url="http://unix/api/v4/internal/allowed"1757gitlab # [ 247.645163] gitaly[910]: time="2026-11-10T10:58:33.071Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF duration_ms=37ns method=POST pid=910 status=200 url="http://unix/api/v4/internal/pre_receive"1758gitlab # [ 247.645418] gitaly[910]: time="2026-11-10T10:58:33.072Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:32.962" grpc.time_ms=109.234 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1759gitlab # [ 247.692242] gitaly[910]: time="2026-11-10T10:58:33.118Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:33.118" grpc.time_ms=0.425 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1760gitlab # [ 247.735943] gitaly[910]: time="2026-11-10T10:58:33.162Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:33.161" grpc.time_ms=0.964 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1761gitlab # [ 247.770105] gitaly[910]: time="2026-11-10T10:58:33.196Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:33.196" grpc.time_ms=0.335 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1762gitlab # [ 247.839166] gitaly[910]: time="2026-11-10T10:58:33.265Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF duration_ms=37ns method=POST pid=910 status=200 url="http://unix/api/v4/internal/post_receive"1763gitlab # [ 247.842202] gitaly[910]: time="2026-11-10T10:58:33.268Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:33.226" grpc.time_ms=42.247 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1764gitlab # [ 247.861653] gitaly[910]: time="2026-11-10T10:58:33.288Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF diskcache=e8a955aa-4d57-4b6a-aa22-7a2a98ceb175 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:32.893" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1765gitlab # [ 247.871157] gitaly[910]: time="2026-11-10T10:58:33.288Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=167 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13586 command.oublock=32 command.real_time_ms=385 command.spawn_token_fork_ms=0 command.system_time_ms=94 command.user_time_ms=73 component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=604 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:32.893" grpc.time_ms=394.47 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1766gitlab # To http://gitlab/root/test-flake.git1767gitlab # b05bdda..fa2bae0 master -> master1768gitlab # [ 247.884540] gitlab-workhorse[823]: time="2026-11-10T10:58:33Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521769gitlab # [ 247.886499] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:33 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 4521770gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.73 seconds)1771gitlab: must succeed: git -C /tmp/test-flake rev-parse master1772gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1773gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1'1774gitlab # [ 247.966877] nixbot[2667]: {"ts": "2026-11-10T10:58:33.392Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:52740 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 HTTP/1.1\" 200"}1775gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1', in 0.07 seconds)1776gitlab # [ 248.023136] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:33.448Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e499a3d4be759d3746b08df6","created_at":"2026-11-10T10:57:38.873Z","trace_propagation_headers":{"sentry-trace":"e9c7281d9fb446cfa177ce075f895c4f-685e03f5c6fb4c95","baggage":"sentry-trace_id=e9c7281d9fb446cfa177ce075f895c4f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPE65R7FES3XZ44646ZPQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:38.911Z","job_size_bytes":2,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e499a3d4be759d3746b08df6: start","job_status":"start","queue_duration_s":54.536797,"scheduling_latency_s":54.536866}1777gitlab # [ 248.264661] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:33.690Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e499a3d4be759d3746b08df6","created_at":"2026-11-10T10:57:38.873Z","trace_propagation_headers":{"sentry-trace":"e9c7281d9fb446cfa177ce075f895c4f-685e03f5c6fb4c95","baggage":"sentry-trace_id=e9c7281d9fb446cfa177ce075f895c4f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPE65R7FES3XZ44646ZPQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:38.911Z","job_size_bytes":2,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e499a3d4be759d3746b08df6: done: 0.242313 sec","job_status":"done","queue_duration_s":54.536797,"scheduling_latency_s":54.536866,"redis_calls":25,"redis_duration_s":0.00851,"redis_read_bytes":912,"redis_write_bytes":2805,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.004728,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002147,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001635,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":11,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.116202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.118330123,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.242313,"completed_at":"2026-11-10T10:58:33.690Z","load_balancing_strategy":"primary","db_duration_s":0.054323,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1778gitlab # [ 248.327903] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:33.692Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"bd2736753ecbfa6a7e23513f","created_at":"2026-11-10T10:57:38.921Z","trace_propagation_headers":{"sentry-trace":"e9c7281d9fb446cfa177ce075f895c4f-685e03f5c6fb4c95","baggage":"sentry-trace_id=e9c7281d9fb446cfa177ce075f895c4f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPE65R7FES3XZ44646ZPQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:38.927Z","job_size_bytes":2,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-bd2736753ecbfa6a7e23513f: start","job_status":"start","queue_duration_s":54.765644,"scheduling_latency_s":54.765678}1779gitlab # [ 248.362863] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:33.789Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"bd2736753ecbfa6a7e23513f","created_at":"2026-11-10T10:57:38.921Z","trace_propagation_headers":{"sentry-trace":"e9c7281d9fb446cfa177ce075f895c4f-685e03f5c6fb4c95","baggage":"sentry-trace_id=e9c7281d9fb446cfa177ce075f895c4f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPE65R7FES3XZ44646ZPQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:38.927Z","job_size_bytes":2,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-bd2736753ecbfa6a7e23513f: done: 0.096514 sec","job_status":"done","queue_duration_s":54.765644,"scheduling_latency_s":54.765678,"redis_calls":14,"redis_duration_s":0.0037809999999999996,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000986,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001577,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001218,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.062913,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.07270784400000001,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.096514,"completed_at":"2026-11-10T10:58:33.789Z","load_balancing_strategy":"primary","db_duration_s":0.023316,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1780gitlab # [ 248.383946] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:33.790Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"fc9342fbc7db6be9089f9af1","created_at":"2026-11-10T10:57:39.469Z","trace_propagation_headers":{"sentry-trace":"e9c7281d9fb446cfa177ce075f895c4f-685e03f5c6fb4c95","baggage":"sentry-trace_id=e9c7281d9fb446cfa177ce075f895c4f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPE65R7FES3XZ44646ZPQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:39.474Z","job_size_bytes":5,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-fc9342fbc7db6be9089f9af1: start","job_status":"start","queue_duration_s":54.316291,"scheduling_latency_s":54.316308}1781gitlab # [ 248.482155] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:33.908Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"fc9342fbc7db6be9089f9af1","created_at":"2026-11-10T10:57:39.469Z","trace_propagation_headers":{"sentry-trace":"e9c7281d9fb446cfa177ce075f895c4f-685e03f5c6fb4c95","baggage":"sentry-trace_id=e9c7281d9fb446cfa177ce075f895c4f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPE65R7FES3XZ44646ZPQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:39.474Z","job_size_bytes":5,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-fc9342fbc7db6be9089f9af1: done: 0.117666 sec","job_status":"done","queue_duration_s":54.316291,"scheduling_latency_s":54.316308,"redis_calls":12,"redis_duration_s":0.002599,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000653,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001564,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.086977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000304229,"duration_s":0.117666,"completed_at":"2026-11-10T10:58:33.908Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024964,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1782gitlab # [ 248.484348] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:33.909Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"737d73dad6c7a6e1c86a8677","created_at":"2026-11-10T10:57:39.498Z","trace_propagation_headers":{"sentry-trace":"e9c7281d9fb446cfa177ce075f895c4f-685e03f5c6fb4c95","baggage":"sentry-trace_id=e9c7281d9fb446cfa177ce075f895c4f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPE65R7FES3XZ44646ZPQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:39.502Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-737d73dad6c7a6e1c86a8677: start","job_status":"start","queue_duration_s":54.40755,"scheduling_latency_s":54.407566}1783gitlab # [ 248.644324] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:34.070Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"737d73dad6c7a6e1c86a8677","created_at":"2026-11-10T10:57:39.498Z","trace_propagation_headers":{"sentry-trace":"e9c7281d9fb446cfa177ce075f895c4f-685e03f5c6fb4c95","baggage":"sentry-trace_id=e9c7281d9fb446cfa177ce075f895c4f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPE65R7FES3XZ44646ZPQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:39.502Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-737d73dad6c7a6e1c86a8677: done: 0.160806 sec","job_status":"done","queue_duration_s":54.40755,"scheduling_latency_s":54.407566,"redis_calls":11,"redis_duration_s":0.002421,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000676,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001294,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000451,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.135,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.135,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.086374,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000284115,"duration_s":0.160806,"completed_at":"2026-11-10T10:58:34.070Z","load_balancing_strategy":"primary","db_duration_s":0.074447,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1784gitlab # [ 248.646636] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:34.071Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"1b23159fc08db213127f5447","created_at":"2026-11-10T10:57:39.504Z","trace_propagation_headers":{"sentry-trace":"e9c7281d9fb446cfa177ce075f895c4f-685e03f5c6fb4c95","baggage":"sentry-trace_id=e9c7281d9fb446cfa177ce075f895c4f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPE65R7FES3XZ44646ZPQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:39.507Z","job_size_bytes":89,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-1b23159fc08db213127f5447: start","job_status":"start","queue_duration_s":54.564019,"scheduling_latency_s":54.564035}1785gitlab # [ 248.720599] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:34.098Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"1b23159fc08db213127f5447","created_at":"2026-11-10T10:57:39.504Z","trace_propagation_headers":{"sentry-trace":"e9c7281d9fb446cfa177ce075f895c4f-685e03f5c6fb4c95","baggage":"sentry-trace_id=e9c7281d9fb446cfa177ce075f895c4f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPE65R7FES3XZ44646ZPQ","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:39.507Z","job_size_bytes":89,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-1b23159fc08db213127f5447: done: 0.026923 sec","job_status":"done","queue_duration_s":54.564019,"scheduling_latency_s":54.564035,"redis_calls":11,"redis_duration_s":0.002327,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000675,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001277,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000375,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000286349,"duration_s":0.026923,"completed_at":"2026-11-10T10:58:34.098Z","load_balancing_strategy":"primary","db_duration_s":0.005264,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1786gitlab # [ 248.729831] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:34.099Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 b05bdda260e45ffb6084becbb5b4f5bb934cf14f refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"cb098b2cf8f14c0f54ef53da","created_at":"2026-11-10T10:57:46.087Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-23e0d49d3a494aaf","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:46.108Z","job_size_bytes":185,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-cb098b2cf8f14c0f54ef53da: start","job_status":"start","queue_duration_s":47.991023,"scheduling_latency_s":47.991039}1787gitlab # [ 248.781948] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:34.111Z","message":"changes: \"0000000000000000000000000000000000000000 b05bdda260e45ffb6084becbb5b4f5bb934cf14f refs/heads/master\\n\""}1788gitlab # [ 248.783382] gitaly[910]: time="2026-11-10T10:58:34.141Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:44.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:34.131" grpc.time_ms=9.484 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1789gitlab # [ 248.784332] gitaly[910]: time="2026-11-10T10:58:34.203Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=50 command.spawn_token_fork_ms=43 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T10:58:44.020" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:58:34.151" grpc.time_ms=51.928 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1790gitlab # [ 248.835784] gitaly[910]: time="2026-11-10T10:58:34.262Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-10T10:58:44.122" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:34.253" grpc.time_ms=9.242 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1791gitlab # [ 248.873174] gitaly[910]: time="2026-11-10T10:58:34.299Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:34.299" grpc.time_ms=0.272 pid=910 span.kind=server system=grpc1792gitlab # [ 248.902112] gitaly[910]: time="2026-11-10T10:58:34.328Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:34.328" grpc.time_ms=0.315 pid=910 span.kind=server system=grpc1793gitlab # [ 248.920761] gitaly[910]: time="2026-11-10T10:58:34.347Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 diskcache=473e9fcc-1ebf-4859-b3a2-46c23ac3d055 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T10:58:44.144" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:58:34.274" pid=910 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1794gitlab # [ 248.921762] gitaly[910]: time="2026-11-10T10:58:34.347Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=67 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5015 command.oublock=8 command.real_time_ms=63 command.spawn_token_fork_ms=0 command.system_time_ms=50 command.user_time_ms=17 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T10:58:44.144" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:58:34.274" grpc.time_ms=72.695 pid=910 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1795gitlab # [ 248.938179] gitaly[910]: time="2026-11-10T10:58:34.364Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-11-10T10:58:44.234" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:58:34.364" grpc.time_ms=0.108 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1796gitlab # [ 248.955390] gitaly[910]: time="2026-11-10T10:58:34.382Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=941 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-11-10T10:58:44.238" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:34.368" grpc.time_ms=13.654 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1797gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1'1798gitlab # [ 249.011236] nixbot[2667]: {"ts": "2026-11-10T10:58:34.437Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:52748 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 HTTP/1.1\" 200"}1799gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1', in 0.04 seconds)1800gitlab # [ 249.029438] gitaly[910]: time="2026-11-10T10:58:34.456Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=338 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-11-10T10:59:04.051" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:34.451" grpc.time_ms=4.251 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1801gitlab # [ 249.049842] gitaly[910]: time="2026-11-10T10:58:34.476Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T10:58:44.005" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:58:34.476" grpc.time_ms=0.291 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1802gitlab # [ 249.064777] gitaly[910]: time="2026-11-10T10:58:34.491Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=341 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-10T10:58:44.017" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-11-10T10:58:34.487" grpc.time_ms=3.938 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1803gitlab # [ 249.086626] gitaly[910]: time="2026-11-10T10:58:34.513Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:44.033" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:34.503" grpc.time_ms=9.448 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1804gitlab # [ 249.101703] gitaly[910]: time="2026-11-10T10:58:34.528Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:44.049" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:34.519" grpc.time_ms=8.917 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1805gitlab # [ 249.122796] gitaly[910]: time="2026-11-10T10:58:34.549Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=963 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-10T10:59:04.140" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:34.540" grpc.time_ms=8.683 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1806gitlab # [ 249.499802] gitaly[910]: time="2026-11-10T10:58:34.926Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-10T10:58:44.004" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:34.924" grpc.time_ms=1.553 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1807gitlab # [ 249.573255] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:34.998Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 b05bdda260e45ffb6084becbb5b4f5bb934cf14f refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"cb098b2cf8f14c0f54ef53da","created_at":"2026-11-10T10:57:46.087Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-23e0d49d3a494aaf","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:57:46.108Z","job_size_bytes":185,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-cb098b2cf8f14c0f54ef53da: done: 0.898407 sec","job_status":"done","queue_duration_s":47.991023,"scheduling_latency_s":47.991039,"gitaly_calls":13,"gitaly_duration_s":0.240582,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.07309099999999999,"redis_read_bytes":2103,"redis_write_bytes":21749,"redis_cache_calls":15,"redis_cache_duration_s":0.003809,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.006337,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001673,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7837,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003338,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.017965,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.039969,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.174,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.196,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.048,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.522257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000504813,"exclusive_lock_hold_duration_s":0.026724905,"duration_s":0.898407,"completed_at":"2026-11-10T10:58:34.998Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.086193,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1808gitlab # [ 249.652161] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:35.078Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"cbd38a5e7d67e90f80404c07","created_at":"2026-11-10T10:57:50.898Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-8cbd6cf49c2f4c14","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:50.903Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-cbd38a5e7d67e90f80404c07: start","job_status":"start","queue_duration_s":44.17447,"scheduling_latency_s":44.174512}1809gitlab # [ 249.740697] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:35.166Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"cbd38a5e7d67e90f80404c07","created_at":"2026-11-10T10:57:50.898Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-8cbd6cf49c2f4c14","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:50.903Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-cbd38a5e7d67e90f80404c07: done: 0.08879 sec","job_status":"done","queue_duration_s":44.17447,"scheduling_latency_s":44.174512,"redis_calls":16,"redis_duration_s":0.004459,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002356,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001552,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000551,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000305066,"duration_s":0.08879,"completed_at":"2026-11-10T10:58:35.166Z","load_balancing_strategy":"primary","db_duration_s":0.023819,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1810gitlab # [ 249.741870] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:35.168Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"07be6a04797432ad15bff38a","created_at":"2026-11-10T10:57:50.904Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-8cbd6cf49c2f4c14","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:50.907Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-07be6a04797432ad15bff38a: start","job_status":"start","queue_duration_s":44.260739,"scheduling_latency_s":44.260756}1811gitlab # [ 249.846082] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:35.271Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"07be6a04797432ad15bff38a","created_at":"2026-11-10T10:57:50.904Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-8cbd6cf49c2f4c14","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:50.907Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-07be6a04797432ad15bff38a: done: 0.103546 sec","job_status":"done","queue_duration_s":44.260739,"scheduling_latency_s":44.260756,"redis_calls":21,"redis_duration_s":0.005062,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.00214,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001552,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00137,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000410946,"duration_s":0.103546,"completed_at":"2026-11-10T10:58:35.271Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032117,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1812gitlab # [ 249.847990] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:35.274Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"557f8ea7015742b03dbcd5e2","created_at":"2026-11-10T10:57:51.014Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-8cbd6cf49c2f4c14","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:51.019Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-557f8ea7015742b03dbcd5e2: start","job_status":"start","queue_duration_s":44.254651,"scheduling_latency_s":44.254679}1813gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1'1814gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1', in 0.02 seconds)1815gitlab # [ 250.038603] nixbot[2667]: {"ts": "2026-11-10T10:58:35.464Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:52760 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 HTTP/1.1\" 200"}1816gitlab # [ 250.054853] gitaly[910]: time="2026-11-10T10:58:35.481Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:35.480" grpc.time_ms=0.496 pid=910 span.kind=server system=grpc1817gitlab # [ 250.100664] gitaly[910]: time="2026-11-10T10:58:35.527Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:35.526" grpc.time_ms=0.387 pid=910 span.kind=server system=grpc1818gitlab # [ 250.147453] gitaly[910]: time="2026-11-10T10:58:35.574Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:35.573" grpc.time_ms=0.491 pid=910 span.kind=server system=grpc1819gitlab # [ 250.164858] gitaly[910]: time="2026-11-10T10:58:35.591Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 diskcache=f3c1dd57-c2d0-434a-9d61-c7ce5eda20d5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T10:59:05.031" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:35.432" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1820gitlab # [ 250.165761] gitaly[910]: time="2026-11-10T10:58:35.591Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=152 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7399 command.oublock=0 command.real_time_ms=151 command.spawn_token_fork_ms=0 command.system_time_ms=103 command.user_time_ms=49 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T10:59:05.031" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:35.432" grpc.time_ms=159.123 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1821gitlab # [ 250.260801] gitaly[910]: time="2026-11-10T10:58:35.687Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T10:59:05.285" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:35.686" grpc.time_ms=1.111 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1822gitlab # [ 250.290196] gitaly[910]: time="2026-11-10T10:58:35.716Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPTB0953NWN9ZTXR22XM5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:05.315" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:35.715" grpc.time_ms=0.944 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1823gitlab # [ 250.330340] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:35.756Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"557f8ea7015742b03dbcd5e2","created_at":"2026-11-10T10:57:51.014Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-8cbd6cf49c2f4c14","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:51.019Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2545,"job_status":"rescheduled","message":"PipelineProcessWorker JID-557f8ea7015742b03dbcd5e2: rescheduled"}1824gitlab # [ 250.344460] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:35.770Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"557f8ea7015742b03dbcd5e2","created_at":"2026-11-10T10:57:51.014Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-8cbd6cf49c2f4c14","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T10:57:51.019Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-557f8ea7015742b03dbcd5e2: done: 0.496407 sec","job_status":"done","queue_duration_s":44.254651,"scheduling_latency_s":44.254679,"gitaly_calls":3,"gitaly_duration_s":0.189613,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.025408,"redis_read_bytes":460,"redis_write_bytes":29292,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001079,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001162,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.008921,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.004109,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15579,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.005726,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000541,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000504,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002287,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.056,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.019,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.022,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.028,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.233408,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.438076907,"duration_s":0.496407,"completed_at":"2026-11-10T10:58:35.770Z","load_balancing_strategy":"primary","db_duration_s":0.068239,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1825gitlab # [ 250.391692] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:35.772Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","b05bdda260e45ffb6084becbb5b4f5bb934cf14f fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"71d28992a2fdce64b7de702b","created_at":"2026-11-10T10:58:33.251Z","trace_propagation_headers":{"sentry-trace":"2bdc67a45a86414dbe01f8a2fef87a19-4fe574a3eea94e93","baggage":"sentry-trace_id=2bdc67a45a86414dbe01f8a2fef87a19,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HR3R3MA3WDV8JDVCXNTVF","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:58:33.252Z","job_size_bytes":185,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-71d28992a2fdce64b7de702b: start","job_status":"start","queue_duration_s":2.519975,"scheduling_latency_s":2.520011}1826gitlab # [ 250.392525] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:35.789Z","message":"changes: \"b05bdda260e45ffb6084becbb5b4f5bb934cf14f fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 refs/heads/master\\n\""}1827gitlab # [ 250.398080] gitaly[910]: time="2026-11-10T10:58:35.824Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:45.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:35.812" grpc.time_ms=11.762 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1828gitlab # [ 250.414517] gitaly[910]: time="2026-11-10T10:58:35.841Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-11-10T10:58:45.020" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:58:35.830" grpc.time_ms=10.226 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1829gitlab # [ 250.420816] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:35.847Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"69072bfa80790bf468b5f3c2","created_at":"2026-11-10T10:58:35.844Z","trace_propagation_headers":{"sentry-trace":"2bdc67a45a86414dbe01f8a2fef87a19-babca15c88fb460e","baggage":"sentry-trace_id=2bdc67a45a86414dbe01f8a2fef87a19,sentry-release=Unknown"},"correlation_id":"01M72HR3R3MA3WDV8JDVCXNTVF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"0efd97b32bffe24857dfed60","job_size_bytes":46,"pid":2545,"job_status":"deduplicated","message":"ProjectCacheWorker JID-69072bfa80790bf468b5f3c2: deduplicated: until executing","deduplication.type":"until executing"}1830gitlab # [ 250.448509] gitaly[910]: time="2026-11-10T10:58:35.875Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-11-10T10:58:45.053" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:35.863" grpc.time_ms=11.322 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1831gitlab # [ 250.465150] gitaly[910]: time="2026-11-10T10:58:35.891Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=650 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-11-10T10:59:05.480" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:35.880" grpc.time_ms=11.137 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1832gitlab # [ 250.481230] gitaly[910]: time="2026-11-10T10:58:35.907Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-11-10T10:58:45.091" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-11-10T10:58:35.902" grpc.time_ms=5.51 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1833gitlab # [ 250.498515] gitaly[910]: time="2026-11-10T10:58:35.925Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:45.106" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:35.916" grpc.time_ms=8.603 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1834gitlab # [ 250.568183] gitaly[910]: time="2026-11-10T10:58:35.994Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-11-10T10:58:45.073" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:35.993" grpc.time_ms=1.085 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1835gitlab # [ 250.610634] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:36.035Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","b05bdda260e45ffb6084becbb5b4f5bb934cf14f fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"71d28992a2fdce64b7de702b","created_at":"2026-11-10T10:58:33.251Z","trace_propagation_headers":{"sentry-trace":"2bdc67a45a86414dbe01f8a2fef87a19-4fe574a3eea94e93","baggage":"sentry-trace_id=2bdc67a45a86414dbe01f8a2fef87a19,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HR3R3MA3WDV8JDVCXNTVF","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-11-10T10:58:33.252Z","job_size_bytes":185,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-71d28992a2fdce64b7de702b: done: 0.262887 sec","job_status":"done","queue_duration_s":2.519975,"scheduling_latency_s":2.520011,"gitaly_calls":7,"gitaly_duration_s":0.090728,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.022025,"redis_read_bytes":501,"redis_write_bytes":11940,"redis_cache_calls":9,"redis_cache_duration_s":0.002487,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000896,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5717,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002865,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.008105,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.007672,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.022,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.022,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.148524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000799543,"exclusive_lock_hold_duration_s":0.022988117,"duration_s":0.262887,"completed_at":"2026-11-10T10:58:36.035Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026278,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1836gitlab # [ 250.620346] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:36.038Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"0efd97b32bffe24857dfed60","created_at":"2026-11-10T10:58:34.207Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-a5e2c5fdf65b482b","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:34.232Z","job_size_bytes":46,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0efd97b32bffe24857dfed60: start","job_status":"start","queue_duration_s":1.805862,"scheduling_latency_s":1.805894}1837gitlab # [ 250.672351] gitaly[910]: time="2026-11-10T10:58:36.078Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-11-10T16:58:36.072" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:58:36.072" grpc.time_ms=5.174 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1838gitlab # [ 250.673154] gitaly[910]: time="2026-11-10T10:58:36.095Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-10T10:59:06.083" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:36.083" grpc.time_ms=11.312 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1839gitlab # [ 250.689782] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:36.116Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"0efd97b32bffe24857dfed60","created_at":"2026-11-10T10:58:34.207Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-a5e2c5fdf65b482b","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:34.232Z","job_size_bytes":46,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0efd97b32bffe24857dfed60: done: 0.078169 sec","job_status":"done","queue_duration_s":1.805862,"scheduling_latency_s":1.805894,"gitaly_calls":2,"gitaly_duration_s":0.020669,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.006775,"redis_read_bytes":69,"redis_write_bytes":4886,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001312,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000607,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2114,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001435,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002744,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000677,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000255339,"duration_s":0.078169,"completed_at":"2026-11-10T10:58:36.116Z","load_balancing_strategy":"primary","db_duration_s":0.012831,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1840gitlab # [ 250.748123] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:36.117Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"dfbd5d5fd35cd62b39b8cfa5","created_at":"2026-11-10T10:58:34.246Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-a5e2c5fdf65b482b","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:34.249Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-dfbd5d5fd35cd62b39b8cfa5: start","job_status":"start","queue_duration_s":1.867739,"scheduling_latency_s":1.867756}1841gitlab # [ 250.748899] gitaly[910]: time="2026-11-10T10:58:36.162Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 diskcache=c60329be-99d2-4875-8277-6e0cec0b17b8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T16:58:36.134" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:36.135" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1842gitlab # [ 250.749701] gitaly[910]: time="2026-11-10T10:58:36.162Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1285 command.oublock=0 command.real_time_ms=27 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T16:58:36.134" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:36.135" grpc.time_ms=26.85 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1843gitlab # [ 250.767229] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:36.193Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"dfbd5d5fd35cd62b39b8cfa5","created_at":"2026-11-10T10:58:34.246Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-a5e2c5fdf65b482b","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:34.249Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-dfbd5d5fd35cd62b39b8cfa5: done: 0.076072 sec","job_status":"done","queue_duration_s":1.867739,"scheduling_latency_s":1.867756,"gitaly_calls":1,"gitaly_duration_s":0.029059,"redis_calls":15,"redis_duration_s":0.003703,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000553,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001315,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000678,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001157,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.017,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.058838484,"duration_s":0.076072,"completed_at":"2026-11-10T10:58:36.193Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016819,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1844gitlab # [ 250.791741] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:36.194Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"cce2b26a20812c06ab268b2d","created_at":"2026-11-10T10:58:34.360Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-a5e2c5fdf65b482b","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:34.362Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-cce2b26a20812c06ab268b2d: start","job_status":"start","queue_duration_s":1.832715,"scheduling_latency_s":1.832756}1845gitlab # [ 250.831182] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:36.226Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"cce2b26a20812c06ab268b2d","created_at":"2026-11-10T10:58:34.360Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-a5e2c5fdf65b482b","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:34.362Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-cce2b26a20812c06ab268b2d: done: 0.031579 sec","job_status":"done","queue_duration_s":1.832715,"scheduling_latency_s":1.832756,"gitaly_calls":1,"gitaly_duration_s":0.008061,"redis_calls":16,"redis_duration_s":0.0034000000000000002,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000233,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001721,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000567,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031579,"completed_at":"2026-11-10T10:58:36.226Z","load_balancing_strategy":"primary","db_duration_s":0.002298,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1846gitlab # [ 250.846247] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:36.227Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"1f51aa7a260837ffe4e31417","created_at":"2026-11-10T10:58:34.786Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-a5e2c5fdf65b482b","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8daf2203bac2cfdbe68f2d0118156755044a8293511cebccea2673895de6cfb7","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:34.792Z","job_size_bytes":467,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-1f51aa7a260837ffe4e31417: start","job_status":"start","queue_duration_s":1.435261,"scheduling_latency_s":1.435278}1847gitlab # [ 250.862118] gitaly[910]: time="2026-11-10T10:58:36.220Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-11-10T10:59:06.213" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:36.213" grpc.time_ms=6.18 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1848gitlab # [ 250.901711] gitaly[910]: time="2026-11-10T10:58:36.246Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:06.245" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:36.245" grpc.time_ms=0.527 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1849gitlab # [ 250.902433] gitaly[910]: time="2026-11-10T10:58:36.256Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T10:58:46.007" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:36.247" grpc.time_ms=8.333 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1850gitlab # [ 250.907575] gitaly[910]: time="2026-11-10T10:58:36.334Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T10:59:06.332" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:36.333" grpc.time_ms=0.931 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1851gitlab # [ 250.912182] gitaly[910]: time="2026-11-10T10:58:36.338Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=61 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-10T10:59:06.336" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:36.336" grpc.time_ms=2.041 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1852gitlab # [ 250.973931] gitaly[910]: time="2026-11-10T10:58:36.400Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:46.153" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:36.394" grpc.time_ms=6.478 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1853gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1'1854gitlab # [ 251.085480] nixbot[2667]: {"ts": "2026-11-10T10:58:36.510Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:52774 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 HTTP/1.1\" 200"}1855gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1', in 0.05 seconds)1856gitlab # [ 251.395407] gitaly[910]: time="2026-11-10T10:58:36.821Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=967 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-10T10:59:06.802" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:36.802" grpc.time_ms=18.319 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1857gitlab # [ 251.410860] gitaly[910]: time="2026-11-10T10:58:36.837Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M72HPNHC58J4F129DPYHBM88 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-10T10:59:06.823" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:36.823" grpc.time_ms=13.859 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1858gitlab # [ 251.474789] sidekiq-cluster[2545]: {"severity":"WARN","time":"2026-11-10T10:58:36.901Z","class":"CreatePipelineWorker","correlation_id":"01M72HPNHC58J4F129DPYHBM88","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"b05bdda260e45ffb6084becbb5b4f5bb934cf14f","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"b05bdda260e45ffb6084becbb5b4f5bb934cf14f","ref":"refs/heads/master"},"retry":0}1859gitlab # [ 251.480292] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:36.906Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"1f51aa7a260837ffe4e31417","created_at":"2026-11-10T10:58:34.786Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-a5e2c5fdf65b482b","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8daf2203bac2cfdbe68f2d0118156755044a8293511cebccea2673895de6cfb7","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:34.792Z","job_size_bytes":467,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-1f51aa7a260837ffe4e31417: done: 0.679125 sec","job_status":"done","queue_duration_s":1.435261,"scheduling_latency_s":1.435278,"gitaly_calls":7,"gitaly_duration_s":0.104973,"redis_calls":51,"redis_duration_s":0.008372,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000549,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.003761,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001167,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002002,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000893,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.517695,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000270984,"duration_s":0.679125,"completed_at":"2026-11-10T10:58:36.906Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.045783,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1860gitlab # [ 251.538981] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:36.908Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"02e4030d-31e6-4906-a500-86fa3da68c70\"}"],"class":"WebHookWorker","jid":"571405e9a3b3a135b1d22f2c","created_at":"2026-11-10T10:58:34.899Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-a5e2c5fdf65b482b","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b1ed9fedcdeec7e6b2743a2dfb94a5c076a10b10241f41f4673e1e566fbe8b4","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:34.904Z","job_size_bytes":1751,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-571405e9a3b3a135b1d22f2c: start","job_status":"start","queue_duration_s":2.003396,"scheduling_latency_s":2.003413}1861gitlab # [ 251.598994] nixbot[2667]: {"ts": "2026-11-10T10:58:37.025Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "8d63dade-0f5f-46ec-9d6f-8317cc799061", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "b05bdda260e45ffb6084becbb5b4f5bb934cf14f", "pr": null}1862gitlab # [ 251.604469] nixbot[2667]: {"ts": "2026-11-10T10:58:37.031Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:52790 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}1863gitlab # [ 251.628337] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.053Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"02e4030d-31e6-4906-a500-86fa3da68c70\"}"],"class":"WebHookWorker","jid":"571405e9a3b3a135b1d22f2c","created_at":"2026-11-10T10:58:34.899Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-a5e2c5fdf65b482b","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b1ed9fedcdeec7e6b2743a2dfb94a5c076a10b10241f41f4673e1e566fbe8b4","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:34.904Z","job_size_bytes":1751,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-571405e9a3b3a135b1d22f2c: done: 0.145675 sec","job_status":"done","queue_duration_s":2.003396,"scheduling_latency_s":2.003413,"redis_calls":19,"redis_duration_s":0.004445,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001151,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000559,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001766,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.06397880200000827,"cpu_s":0.067353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000396419,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.145675,"completed_at":"2026-11-10T10:58:37.053Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1864gitlab # [ 251.676979] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.103Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3987f83af2b5dff58d3ba22c","created_at":"2026-11-10T10:58:35.420Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.424Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3987f83af2b5dff58d3ba22c: start","job_status":"start","queue_duration_s":1.678842,"scheduling_latency_s":1.678891}1865gitlab # [ 251.701363] gitaly[910]: time="2026-11-10T10:58:37.127Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=8 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72HR7W6S9ZDH3V062M5Y9VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:37.110" grpc.time_ms=16.28 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1866gitlab # [ 251.713756] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:37 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 641867gitlab # [ 251.767280] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.193Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3987f83af2b5dff58d3ba22c","created_at":"2026-11-10T10:58:35.420Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.424Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3987f83af2b5dff58d3ba22c: done: 0.090688 sec","job_status":"done","queue_duration_s":1.678842,"scheduling_latency_s":1.678891,"redis_calls":10,"redis_duration_s":0.004046,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000606,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002609,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000831,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.062081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000532749,"duration_s":0.090688,"completed_at":"2026-11-10T10:58:37.193Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012255,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1868gitlab # [ 251.788238] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.195Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a6b5f30b4c3f2aac4a2f2ba0","created_at":"2026-11-10T10:58:35.598Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.603Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a6b5f30b4c3f2aac4a2f2ba0: start","job_status":"start","queue_duration_s":1.591368,"scheduling_latency_s":1.591808}1869gitlab # [ 251.789518] gitlab-workhorse[823]: time="2026-11-10T10:58:37Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671870gitlab # [ 251.790376] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:37 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 711871gitlab # [ 251.790870] gitaly[910]: time="2026-11-10T10:58:37.198Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72HR7Y9DKWGCNQW9AR9FRQF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:37.189" pid=910 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1872gitlab # [ 251.794602] gitaly[910]: time="2026-11-10T10:58:37.198Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR7Y9DKWGCNQW9AR9FRQF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:37.189" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1873gitlab # [ 251.806259] gitaly[910]: time="2026-11-10T10:58:37.199Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR7Y9DKWGCNQW9AR9FRQF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:37.189" grpc.time_ms=10.106 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1874gitlab # [ 251.826349] gitaly[910]: time="2026-11-10T10:58:37.252Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72HR80JM59TVB4QHR1KMYB1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:37.238" pid=910 remote_ip="2001:db8:1::1" request_sha=3f08cc9b6642260b56860b51cbcf3822acdc67f2 response_bytes=337 span.kind=server system=grpc1875gitlab # [ 251.828635] gitaly[910]: time="2026-11-10T10:58:37.252Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1196 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR80JM59TVB4QHR1KMYB1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:37.238" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1876gitlab # [ 251.841023] gitlab-workhorse[823]: time="2026-11-10T10:58:37Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3371877gitlab # [ 251.842692] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:37 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 511878gitlab # [ 251.844406] gitaly[910]: time="2026-11-10T10:58:37.253Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1196 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR80JM59TVB4QHR1KMYB1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:37.238" grpc.time_ms=14.781 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1879gitlab # [ 251.889578] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.309Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a6b5f30b4c3f2aac4a2f2ba0","created_at":"2026-11-10T10:58:35.598Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.603Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a6b5f30b4c3f2aac4a2f2ba0: done: 0.114567 sec","job_status":"done","queue_duration_s":1.591368,"scheduling_latency_s":1.591808,"redis_calls":10,"redis_duration_s":0.004869,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002307,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0022,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000362,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.084,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.084,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000243327,"duration_s":0.114567,"completed_at":"2026-11-10T10:58:37.309Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.084183,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1880gitlab # [ 251.898508] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.311Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"ec3a22133232fa19c1f6e29e","created_at":"2026-11-10T10:58:35.615Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.623Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-ec3a22133232fa19c1f6e29e: start","job_status":"start","queue_duration_s":1.687473,"scheduling_latency_s":1.68749}1881gitlab # [ 251.940221] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.340Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"ec3a22133232fa19c1f6e29e","created_at":"2026-11-10T10:58:35.615Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.623Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-ec3a22133232fa19c1f6e29e: done: 0.02949 sec","job_status":"done","queue_duration_s":1.687473,"scheduling_latency_s":1.68749,"redis_calls":10,"redis_duration_s":0.002657,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001252,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001145,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000228801,"duration_s":0.02949,"completed_at":"2026-11-10T10:58:37.340Z","load_balancing_strategy":"primary","db_duration_s":0.004879,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1882gitlab # [ 251.951587] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.341Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8599bc546d072db1e3019bfc","created_at":"2026-11-10T10:58:35.635Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.638Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8599bc546d072db1e3019bfc: start","job_status":"start","queue_duration_s":1.702493,"scheduling_latency_s":1.702508}1883gitlab # [ 251.990163] nixbot[2667]: {"ts": "2026-11-10T10:58:37.323Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1884gitlab # [ 251.990820] nixbot[2667]: {"ts": "2026-11-10T10:58:37.325Z", "level": "info", "logger": "nixbot.after_build", "msg": "reused build: replaying effect statuses", "build_id": 1, "sha": "b05bdda260e45ffb6084becbb5b4f5bb934cf14f"}1885gitlab # [ 251.991457] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.364Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8599bc546d072db1e3019bfc","created_at":"2026-11-10T10:58:35.635Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.638Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8599bc546d072db1e3019bfc: done: 0.023717 sec","job_status":"done","queue_duration_s":1.702493,"scheduling_latency_s":1.702508,"redis_calls":13,"redis_duration_s":0.001498,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000334,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000227,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000194718,"duration_s":0.023717,"completed_at":"2026-11-10T10:58:37.364Z","load_balancing_strategy":"primary","db_duration_s":0.003182,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1886gitlab # [ 252.042072] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.365Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"2001c1e56ed40e87cef3a4ed","created_at":"2026-11-10T10:58:35.639Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.642Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-2001c1e56ed40e87cef3a4ed: start","job_status":"start","queue_duration_s":1.72369,"scheduling_latency_s":1.723705}1887gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1'1888gitlab # [ 252.042762] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.386Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"2001c1e56ed40e87cef3a4ed","created_at":"2026-11-10T10:58:35.639Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.642Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-2001c1e56ed40e87cef3a4ed: done: 0.021209 sec","job_status":"done","queue_duration_s":1.72369,"scheduling_latency_s":1.723705,"redis_calls":10,"redis_duration_s":0.001232,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000191645,"duration_s":0.021209,"completed_at":"2026-11-10T10:58:37.386Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003425,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1889gitlab # [ 252.085799] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.387Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"0d8aee92341e9e6dc3198c65","created_at":"2026-11-10T10:58:35.642Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.646Z","job_size_bytes":19,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-0d8aee92341e9e6dc3198c65: start","job_status":"start","queue_duration_s":1.741641,"scheduling_latency_s":1.741676}1890gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1', in 0.04 seconds)1891gitlab # [ 252.086611] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.406Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"0d8aee92341e9e6dc3198c65","created_at":"2026-11-10T10:58:35.642Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.646Z","job_size_bytes":19,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-0d8aee92341e9e6dc3198c65: done: 0.018951 sec","job_status":"done","queue_duration_s":1.741641,"scheduling_latency_s":1.741676,"redis_calls":11,"redis_duration_s":0.001332,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000228,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000225,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000207848,"duration_s":0.018951,"completed_at":"2026-11-10T10:58:37.406Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002412,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1892gitlab # [ 252.138920] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.407Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"4bd22ea55051184ff5e03893","created_at":"2026-11-10T10:58:35.648Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.651Z","job_size_bytes":66,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-4bd22ea55051184ff5e03893: start","job_status":"start","queue_duration_s":1.756658,"scheduling_latency_s":1.756672}1893gitlab # [ 252.139559] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.428Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"4bd22ea55051184ff5e03893","created_at":"2026-11-10T10:58:35.648Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.651Z","job_size_bytes":66,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-4bd22ea55051184ff5e03893: done: 0.020355 sec","job_status":"done","queue_duration_s":1.756658,"scheduling_latency_s":1.756672,"redis_calls":10,"redis_duration_s":0.001233,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000234,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000210362,"duration_s":0.020355,"completed_at":"2026-11-10T10:58:37.428Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003063,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1894gitlab # [ 252.191019] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.429Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"2cfa69fccec348eff8d4ddf2","created_at":"2026-11-10T10:58:35.651Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.655Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-2cfa69fccec348eff8d4ddf2: start","job_status":"start","queue_duration_s":1.774009,"scheduling_latency_s":1.774024}1895gitlab # [ 252.192089] nixbot[2667]: {"ts": "2026-11-10T10:58:37.555Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:52800 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 HTTP/1.1\" 200"}1896gitlab # [ 252.192572] gitaly[910]: time="2026-11-10T10:58:37.584Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR8B34T5MGSM3WGZN9A72 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:07.483" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:37.583" grpc.time_ms=0.541 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1897gitlab # [ 252.235943] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.453Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"2cfa69fccec348eff8d4ddf2","created_at":"2026-11-10T10:58:35.651Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.655Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-2cfa69fccec348eff8d4ddf2: done: 0.02494 sec","job_status":"done","queue_duration_s":1.774009,"scheduling_latency_s":1.774024,"redis_calls":10,"redis_duration_s":0.001186,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000229,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000731,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000226,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000217067,"duration_s":0.02494,"completed_at":"2026-11-10T10:58:37.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1898gitlab # [ 252.243505] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.454Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"e3e025e235b66eed282fe4b3","created_at":"2026-11-10T10:58:35.659Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.662Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-e3e025e235b66eed282fe4b3: start","job_status":"start","queue_duration_s":1.792615,"scheduling_latency_s":1.79263}1899gitlab # [ 252.282222] gitaly[910]: time="2026-11-10T10:58:37.630Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR8B34T5MGSM3WGZN9A72 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:47.225" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:37.625" grpc.time_ms=4.453 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900gitlab # [ 252.282700] gitaly[910]: time="2026-11-10T10:58:37.632Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF diskcache=57a3c70f-bb35-4db7-a400-eef207ecb222 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T16:58:37.615" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:37.616" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1901gitlab # [ 252.283210] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b05bdda260e45ffb6084becbb5b4f5bb934cf14f HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 1241902gitlab # [ 252.283692] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.478Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"e3e025e235b66eed282fe4b3","created_at":"2026-11-10T10:58:35.659Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.662Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-e3e025e235b66eed282fe4b3: done: 0.023209 sec","job_status":"done","queue_duration_s":1.792615,"scheduling_latency_s":1.79263,"redis_calls":10,"redis_duration_s":0.001238,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000232,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000728,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000278,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000194996,"duration_s":0.023209,"completed_at":"2026-11-10T10:58:37.478Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005924,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1903gitlab # [ 252.329526] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.479Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3253f0cab8b3061da85d403d","created_at":"2026-11-10T10:58:35.662Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.665Z","job_size_bytes":6,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3253f0cab8b3061da85d403d: start","job_status":"start","queue_duration_s":1.813295,"scheduling_latency_s":1.81331}1904gitlab # [ 252.334582] gitaly[910]: time="2026-11-10T10:58:37.632Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=6 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-11-10T16:58:37.615" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:37.616" grpc.time_ms=15.814 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1905gitlab # [ 252.377048] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.493Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3253f0cab8b3061da85d403d","created_at":"2026-11-10T10:58:35.662Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.665Z","job_size_bytes":6,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3253f0cab8b3061da85d403d: done: 0.014594 sec","job_status":"done","queue_duration_s":1.813295,"scheduling_latency_s":1.81331,"redis_calls":10,"redis_duration_s":0.0012079999999999999,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000268,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000712,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000228,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000216508,"duration_s":0.014594,"completed_at":"2026-11-10T10:58:37.493Z","load_balancing_strategy":"primary","db_duration_s":0.001484,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1906gitlab # [ 252.421268] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.494Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"4a7d5ad7d92946b14c0a2301","created_at":"2026-11-10T10:58:35.666Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.668Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-4a7d5ad7d92946b14c0a2301: start","job_status":"start","queue_duration_s":1.825565,"scheduling_latency_s":1.825579}1907gitlab # [ 252.422241] gitaly[910]: time="2026-11-10T10:58:37.660Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:07.659" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:37.659" grpc.time_ms=0.424 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1908gitlab # [ 252.422908] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b05bdda260e45ffb6084becbb5b4f5bb934cf14f HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1311909gitlab # [ 252.423696] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.523Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"4a7d5ad7d92946b14c0a2301","created_at":"2026-11-10T10:58:35.666Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.668Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-4a7d5ad7d92946b14c0a2301: done: 0.028485 sec","job_status":"done","queue_duration_s":1.825565,"scheduling_latency_s":1.825579,"redis_calls":10,"redis_duration_s":0.004141,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000225,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002948,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000835582,"duration_s":0.028485,"completed_at":"2026-11-10T10:58:37.522Z","load_balancing_strategy":"primary","db_duration_s":0.007131,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1910gitlab # [ 252.480048] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.524Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"2a86071aab99b0823175f87d","created_at":"2026-11-10T10:58:35.673Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.676Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-2a86071aab99b0823175f87d: start","job_status":"start","queue_duration_s":1.847739,"scheduling_latency_s":1.847768}1911gitlab # [ 252.480815] gitaly[910]: time="2026-11-10T10:58:37.667Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T10:58:47.001" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:37.661" grpc.time_ms=5.302 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1912gitlab # [ 252.481654] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b05bdda260e45ffb6084becbb5b4f5bb934cf14f HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 911913gitlab # [ 252.482198] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.576Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"2a86071aab99b0823175f87d","created_at":"2026-11-10T10:58:35.673Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.676Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-2a86071aab99b0823175f87d: done: 0.052172 sec","job_status":"done","queue_duration_s":1.847739,"scheduling_latency_s":1.847768,"redis_calls":14,"redis_duration_s":0.002197,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000356749,"duration_s":0.052172,"completed_at":"2026-11-10T10:58:37.576Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012419,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1914gitlab # [ 252.528019] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.577Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"417065545c47c04c29f73847","created_at":"2026-11-10T10:58:35.757Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.759Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-417065545c47c04c29f73847: start","job_status":"start","queue_duration_s":1.818168,"scheduling_latency_s":1.818185}1915gitlab # [ 252.534779] gitaly[910]: time="2026-11-10T10:58:37.695Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T10:59:07.693" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:37.694" grpc.time_ms=0.504 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1916gitlab # [ 252.545167] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.600Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"417065545c47c04c29f73847","created_at":"2026-11-10T10:58:35.757Z","trace_propagation_headers":{"sentry-trace":"3805261ca8ba4907a004f21bfa58ea7b-ad815c49caf94776","baggage":"sentry-trace_id=3805261ca8ba4907a004f21bfa58ea7b,sentry-release=Unknown"},"correlation_id":"01M72HPTB0953NWN9ZTXR22XM5","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.759Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-417065545c47c04c29f73847: done: 0.022724 sec","job_status":"done","queue_duration_s":1.818168,"scheduling_latency_s":1.818185,"redis_calls":8,"redis_duration_s":0.002544,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002277,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000267,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022724,"completed_at":"2026-11-10T10:58:37.600Z","load_balancing_strategy":"primary","db_duration_s":0.005663,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1917gitlab # [ 252.601703] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.601Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"2ca81ba7f631ab2fffd24fc3","created_at":"2026-11-10T10:58:35.855Z","trace_propagation_headers":{"sentry-trace":"2bdc67a45a86414dbe01f8a2fef87a19-babca15c88fb460e","baggage":"sentry-trace_id=2bdc67a45a86414dbe01f8a2fef87a19,sentry-release=Unknown"},"correlation_id":"01M72HR3R3MA3WDV8JDVCXNTVF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"dfbd5d5fd35cd62b39b8cfa5","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.859Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-2ca81ba7f631ab2fffd24fc3: start","job_status":"start","queue_duration_s":1.742152,"scheduling_latency_s":1.742169}1918gitlab # [ 252.602603] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:37 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 381919gitlab # [ 252.604656] gitlab-workhorse[823]: time="2026-11-10T10:58:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671920gitlab # [ 252.605435] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 411921gitlab # [ 252.605929] gitaly[910]: time="2026-11-10T10:58:37.728Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR8F9MXE77VEWKQP5HF3E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:07.725" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:37.725" grpc.time_ms=2.678 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1922gitlab # [ 252.651131] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.643Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"2ca81ba7f631ab2fffd24fc3","created_at":"2026-11-10T10:58:35.855Z","trace_propagation_headers":{"sentry-trace":"2bdc67a45a86414dbe01f8a2fef87a19-babca15c88fb460e","baggage":"sentry-trace_id=2bdc67a45a86414dbe01f8a2fef87a19,sentry-release=Unknown"},"correlation_id":"01M72HR3R3MA3WDV8JDVCXNTVF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"dfbd5d5fd35cd62b39b8cfa5","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.859Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-2ca81ba7f631ab2fffd24fc3: done: 0.042159 sec","job_status":"done","queue_duration_s":1.742152,"scheduling_latency_s":1.742169,"gitaly_calls":1,"gitaly_duration_s":0.017471,"redis_calls":12,"redis_duration_s":0.002955,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000356,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001628,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.027290898,"duration_s":0.042159,"completed_at":"2026-11-10T10:58:37.643Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003044,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1923gitlab # [ 252.655792] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:37.644Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2c7720abebe514229d809ffe","created_at":"2026-11-10T10:58:35.971Z","trace_propagation_headers":{"sentry-trace":"2bdc67a45a86414dbe01f8a2fef87a19-babca15c88fb460e","baggage":"sentry-trace_id=2bdc67a45a86414dbe01f8a2fef87a19,sentry-release=Unknown"},"correlation_id":"01M72HR3R3MA3WDV8JDVCXNTVF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:64e988d1ee52d24cce7daef0df1394b66cf8d22e6283a1ebd2d52a56f44cd77a","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.972Z","job_size_bytes":467,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2c7720abebe514229d809ffe: start","job_status":"start","queue_duration_s":1.671963,"scheduling_latency_s":1.67198}1924gitlab # [ 252.698689] gitaly[910]: time="2026-11-10T10:58:37.768Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72HR8F9MXE77VEWKQP5HF3E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:47.213" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:37.764" grpc.time_ms=4.753 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1925gitlab # [ 252.699513] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 451926gitlab # [ 252.699982] gitaly[910]: time="2026-11-10T10:58:37.791Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=324 command.oublock=0 command.real_time_ms=93 command.spawn_token_fork_ms=90 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-11-10T10:59:07.697" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:37.697" grpc.time_ms=93.973 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1927gitlab # [ 252.700580] gitaly[910]: time="2026-11-10T10:58:37.821Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:47.155" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:37.815" grpc.time_ms=5.404 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1928gitlab # [ 252.711169] gitaly[910]: time="2026-11-10T10:58:37.845Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR8KMV4W17Y1V596WZCST grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:07.844" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:37.844" grpc.time_ms=0.413 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1929gitlab # [ 252.722603] gitaly[910]: time="2026-11-10T10:58:37.873Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR8KMV4W17Y1V596WZCST grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:47.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:37.865" grpc.time_ms=8.082 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1930gitlab # [ 252.771539] gitlab-workhorse[823]: time="2026-11-10T10:58:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671931gitlab # [ 252.772158] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 571932gitlab # [ 252.772616] gitaly[910]: time="2026-11-10T10:58:37.966Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR8Q7Y7B8ZJZWDAYYPDKW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:37.959" grpc.time_ms=6.698 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1933gitlab # [ 252.774184] gitaly[910]: time="2026-11-10T10:58:38.008Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72HR8RGNVHECVPKK303CAJ2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:38.002" pid=910 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1934gitlab # [ 252.774864] gitaly[910]: time="2026-11-10T10:58:38.008Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR8RGNVHECVPKK303CAJ2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:38.002" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1935gitlab # [ 252.775493] gitaly[910]: time="2026-11-10T10:58:38.009Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR8RGNVHECVPKK303CAJ2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:38.002" grpc.time_ms=6.393 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1936gitlab # [ 252.783640] gitaly[910]: time="2026-11-10T10:58:38.080Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR8TKXDA8XHSZ5W4Q13RQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:38.071" grpc.time_ms=8.246 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1937gitlab # [ 252.795415] gitaly[910]: time="2026-11-10T10:58:38.139Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72HR8W132MWWY4D4RH6GD6T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:38.129" pid=910 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1938gitlab # [ 252.804543] sidekiq-cluster[2545]: {"severity":"WARN","time":"2026-11-10T10:58:38.202Z","class":"CreatePipelineWorker","correlation_id":"01M72HR3R3MA3WDV8JDVCXNTVF","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"b05bdda260e45ffb6084becbb5b4f5bb934cf14f","after":"fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1","ref":"refs/heads/master"},"retry":0}1939gitlab # [ 252.848921] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:38.207Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2c7720abebe514229d809ffe","created_at":"2026-11-10T10:58:35.971Z","trace_propagation_headers":{"sentry-trace":"2bdc67a45a86414dbe01f8a2fef87a19-babca15c88fb460e","baggage":"sentry-trace_id=2bdc67a45a86414dbe01f8a2fef87a19,sentry-release=Unknown"},"correlation_id":"01M72HR3R3MA3WDV8JDVCXNTVF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:64e988d1ee52d24cce7daef0df1394b66cf8d22e6283a1ebd2d52a56f44cd77a","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.972Z","job_size_bytes":467,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2c7720abebe514229d809ffe: done: 0.562886 sec","job_status":"done","queue_duration_s":1.671963,"scheduling_latency_s":1.67198,"gitaly_calls":7,"gitaly_duration_s":0.134865,"redis_calls":33,"redis_duration_s":0.004555,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000682,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000987,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002153,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000733,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.403772,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.562886,"completed_at":"2026-11-10T10:58:38.207Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017521,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1940gitlab # [ 252.893395] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 511941gitlab # [ 252.894060] gitaly[910]: time="2026-11-10T10:58:38.139Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=7 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR8W132MWWY4D4RH6GD6T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:38.129" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1942gitlab # [ 252.894766] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:38.209Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"dc745a4a-e545-484e-8c4d-b95098fd6afe\"}"],"class":"WebHookWorker","jid":"ab4a35ac5f2728563965b525","created_at":"2026-11-10T10:58:35.983Z","trace_propagation_headers":{"sentry-trace":"2bdc67a45a86414dbe01f8a2fef87a19-babca15c88fb460e","baggage":"sentry-trace_id=2bdc67a45a86414dbe01f8a2fef87a19,sentry-release=Unknown"},"correlation_id":"01M72HR3R3MA3WDV8JDVCXNTVF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:87c3d7f8d33f743deecf9ebb2d4e26ead1fe4637a55fc887dea20015d14feece","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.984Z","job_size_bytes":1749,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-ab4a35ac5f2728563965b525: start","job_status":"start","queue_duration_s":2.224751,"scheduling_latency_s":2.22481}1943gitlab # [ 252.896416] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:38.249Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"dc745a4a-e545-484e-8c4d-b95098fd6afe\"}"],"class":"WebHookWorker","jid":"ab4a35ac5f2728563965b525","created_at":"2026-11-10T10:58:35.983Z","trace_propagation_headers":{"sentry-trace":"2bdc67a45a86414dbe01f8a2fef87a19-babca15c88fb460e","baggage":"sentry-trace_id=2bdc67a45a86414dbe01f8a2fef87a19,sentry-release=Unknown"},"correlation_id":"01M72HR3R3MA3WDV8JDVCXNTVF","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:87c3d7f8d33f743deecf9ebb2d4e26ead1fe4637a55fc887dea20015d14feece","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:35.984Z","job_size_bytes":1749,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-ab4a35ac5f2728563965b525: done: 0.040169 sec","job_status":"done","queue_duration_s":2.224751,"scheduling_latency_s":2.22481,"redis_calls":14,"redis_duration_s":0.0033510000000000002,"redis_read_bytes":25,"redis_write_bytes":4683,"redis_queues_calls":2,"redis_queues_duration_s":0.000496,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3727,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002151,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000704,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.008663392000016756,"cpu_s":0.017472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.040169,"completed_at":"2026-11-10T10:58:38.249Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002195,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1944gitlab # [ 252.936506] nixbot[2667]: {"ts": "2026-11-10T10:58:38.237Z", "level": "info", "logger": "nixbot.webhooks", "msg": "webhook accepted", "delivery": "522db5b3-0ba2-40a8-aaad-aa20acec1d9b", "event": "ChangeRequest", "forge": "gitlab", "repo_id": "1", "branch": "master", "sha": "fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1", "pr": null}1945gitlab # [ 252.937195] nixbot[2667]: {"ts": "2026-11-10T10:58:38.240Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:52810 - \"POST /webhooks/gitlab HTTP/1.1\" 202"}1946gitlab # [ 252.937655] gitaly[910]: time="2026-11-10T10:58:38.139Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=7 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR8W132MWWY4D4RH6GD6T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:38.129" grpc.time_ms=9.849 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1947gitlab # [ 252.983236] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:38.250Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"1c6707d4c214d96e7419992b","created_at":"2026-11-10T10:58:36.104Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-0548daa0cc624a68","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:36.107Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-1c6707d4c214d96e7419992b: start","job_status":"start","queue_duration_s":2.143178,"scheduling_latency_s":2.143196}1948gitlab # [ 252.983943] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:38.288Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"1c6707d4c214d96e7419992b","created_at":"2026-11-10T10:58:36.104Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-0548daa0cc624a68","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:36.107Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-1c6707d4c214d96e7419992b: done: 0.037589 sec","job_status":"done","queue_duration_s":2.143178,"scheduling_latency_s":2.143196,"redis_calls":19,"redis_duration_s":0.007251,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001501,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.001557,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003617,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000576,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.010153525,"duration_s":0.037589,"completed_at":"2026-11-10T10:58:38.288Z","load_balancing_strategy":"primary","db_duration_s":0.004548,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1949gitlab # [ 253.026874] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 661950gitlab # [ 253.027769] gitlab-workhorse[823]: time="2026-11-10T10:58:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2161951gitlab # [ 253.028460] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 1841952gitlab # [ 253.028942] gitlab-workhorse[823]: time="2026-11-10T10:58:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671953gitlab # [ 253.029546] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1121954gitlab # [ 253.030117] gitaly[910]: time="2026-11-10T10:58:38.180Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-11-10T10:59:08.173" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:38.173" grpc.time_ms=7.385 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1955gitlab # [ 253.030759] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:38.289Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","571405e9a3b3a135b1d22f2c"],"class":"WebHooks::LogExecutionWorker","jid":"947d905b8479cafd8f99fc84","created_at":"2026-11-10T10:58:37.035Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-0935c741721d487c","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8cdcedf6e5abf4001a43d0da2ab77f79b4d962468a8d82f8d4be336c7cc5552e","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:37.043Z","job_size_bytes":2593,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-947d905b8479cafd8f99fc84: start","job_status":"start","queue_duration_s":1.245986,"scheduling_latency_s":1.246001}1956gitlab # [ 253.039835] gitaly[910]: time="2026-11-10T10:58:38.191Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR3R3MA3WDV8JDVCXNTVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-11-10T10:59:08.183" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:38.183" grpc.time_ms=7.889 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1957gitlab # [ 253.084753] gitaly[910]: time="2026-11-10T10:58:38.220Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=9 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR8YS7VQVHQQNCGHAVCT7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:38.205" grpc.time_ms=15.008 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1958gitlab # [ 253.085799] gitaly[910]: time="2026-11-10T10:58:38.322Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=5 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M72HR91GHR88F6ZZCN5E5HJN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:38.305" grpc.time_ms=16.812 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1959gitlab # [ 253.086558] gitaly[910]: time="2026-11-10T10:58:38.406Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72HR90EQ36ZVKHNA5MJD766 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:38.348" pid=910 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc1960gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1'1961gitlab # [ 253.087417] gitaly[910]: time="2026-11-10T10:58:38.406Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1165 command.oublock=0 command.real_time_ms=56 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR90EQ36ZVKHNA5MJD766 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:38.348" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1962gitlab # [ 253.097295] gitaly[910]: time="2026-11-10T10:58:38.407Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1165 command.oublock=0 command.real_time_ms=56 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR90EQ36ZVKHNA5MJD766 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:38.348" grpc.time_ms=58.712 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1963gitlab # [ 253.110953] gitaly[910]: time="2026-11-10T10:58:38.436Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M72HR93MH45QQ6D513BCSWZH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:38.431" pid=910 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1964gitlab # [ 253.120709] gitaly[910]: time="2026-11-10T10:58:38.437Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR93MH45QQ6D513BCSWZH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:38.431" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1965gitlab # [ 253.162109] gitaly[910]: time="2026-11-10T10:58:38.437Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR93MH45QQ6D513BCSWZH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-11-10T10:58:38.431" grpc.time_ms=5.809 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc1966gitlab # [ 253.162783] gitaly[910]: time="2026-11-10T10:58:38.497Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR97SB98R6MWKRW8R2XMY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:08.396" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:38.496" grpc.time_ms=0.479 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1967gitlab # [ 253.163428] gitaly[910]: time="2026-11-10T10:58:38.507Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR97SB98R6MWKRW8R2XMY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-11-10T10:59:08.402" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:38.502" grpc.time_ms=4.976 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1968gitlab # [ 253.168238] gitaly[910]: time="2026-11-10T10:58:38.517Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR97SB98R6MWKRW8R2XMY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:48.111" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:38.511" grpc.time_ms=5.495 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1969gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1', in 0.13 seconds)1970gitlab # [ 253.181395] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:38.594Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","571405e9a3b3a135b1d22f2c"],"class":"WebHooks::LogExecutionWorker","jid":"947d905b8479cafd8f99fc84","created_at":"2026-11-10T10:58:37.035Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-0935c741721d487c","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8cdcedf6e5abf4001a43d0da2ab77f79b4d962468a8d82f8d4be336c7cc5552e","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:37.043Z","job_size_bytes":2593,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-947d905b8479cafd8f99fc84: done: 0.304506 sec","job_status":"done","queue_duration_s":1.245986,"scheduling_latency_s":1.246001,"redis_calls":10,"redis_duration_s":0.267745,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000257,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.267113,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000375,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.263190992,"duration_s":0.304506,"completed_at":"2026-11-10T10:58:38.594Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005855,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1971(finished: subtest: push delivers the webhook, in 48.51 seconds)1972subtest: eval and build post commit statuses1973gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses1974gitlab # [ 253.247879] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:38.595Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"fcee583f88a2b7e76855764b","created_at":"2026-11-10T10:58:37.651Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-6aff6339aa2f4a7a","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:37.654Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fcee583f88a2b7e76855764b: start","job_status":"start","queue_duration_s":0.940933,"scheduling_latency_s":0.940952}1975gitlab # [ 253.248711] gitaly[910]: time="2026-11-10T10:58:38.537Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M72HR97SB98R6MWKRW8R2XMY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-11-10T10:58:48.136" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:58:38.536" grpc.time_ms=0.195 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1976gitlab # [ 253.249705] nixbot[2667]: {"ts": "2026-11-10T10:58:38.609Z", "level": "info", "logger": "uvicorn.access", "msg": "::ffff:127.0.0.1:41532 - \"GET /api/repos/gitlab/root/test-flake/builds?commit=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 HTTP/1.1\" 200"}1977gitlab # [ 253.250309] gitaly[910]: time="2026-11-10T10:58:38.544Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR97SB98R6MWKRW8R2XMY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-11-10T10:58:48.139" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:38.540" grpc.time_ms=4.43 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1978gitlab # [ 253.260248] gitaly[910]: time="2026-11-10T10:58:38.569Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HR97SB98R6MWKRW8R2XMY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:38.569" grpc.time_ms=0.304 pid=910 span.kind=server system=grpc1979gitlab # [ 253.297994] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 2261980gitlab # [ 253.298619] nixbot[2667]: {"ts": "2026-11-10T10:58:38.693Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-fa2bae0b1e46-6"}1981gitlab # [ 253.299132] gitaly[910]: time="2026-11-10T10:58:38.588Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HR97SB98R6MWKRW8R2XMY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:38.587" grpc.time_ms=0.333 pid=910 span.kind=server system=grpc1982gitlab # [ 253.299726] gitaly[910]: time="2026-11-10T10:58:38.614Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72HR97SB98R6MWKRW8R2XMY diskcache=9bc491b1-23db-43de-af84-eb6157a30179 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T10:58:48.145" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:58:38.546" pid=910 ref=refs/keep-around/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 remote_ip="2001:db8:1::1" revision=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 span.kind=server system=grpc username=root1983gitlab # [ 253.300547] gitaly[910]: time="2026-11-10T10:58:38.614Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=42 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5278 command.oublock=8 command.real_time_ms=57 command.spawn_token_fork_ms=0 command.system_time_ms=30 command.user_time_ms=12 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR97SB98R6MWKRW8R2XMY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-11-10T10:58:48.145" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-11-10T10:58:38.546" grpc.time_ms=68.453 pid=910 ref=refs/keep-around/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 remote_ip="2001:db8:1::1" revision=fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 span.kind=server system=grpc username=root1984gitlab # [ 253.337912] gitaly[910]: time="2026-11-10T10:58:38.619Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR97SB98R6MWKRW8R2XMY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:08.518" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:38.619" grpc.time_ms=0.277 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1985gitlab # [ 253.338654] gitaly[910]: time="2026-11-10T10:58:38.698Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HR8B34T5MGSM3WGZN9A72 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:38.698" grpc.time_ms=0.441 pid=910 span.kind=server system=grpc1986gitlab # [ 253.339218] gitaly[910]: time="2026-11-10T10:58:38.721Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HR8B34T5MGSM3WGZN9A72 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:38.720" grpc.time_ms=0.273 pid=910 span.kind=server system=grpc1987gitlab # [ 253.350990] gitaly[910]: time="2026-11-10T10:58:38.777Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR9GP5C8T5F56A8CC21F8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:08.676" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:38.777" grpc.time_ms=0.431 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1988gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses, in 0.12 seconds)1989gitlab # [ 253.356613] nixbot[2667]: {"ts": "2026-11-10T10:58:38.783Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-fa2bae0b1e46-6", "argv0": "bwrap"}1990gitlab # [ 253.372575] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 551991gitlab # [ 253.393932] gitaly[910]: time="2026-11-10T10:58:38.820Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HR8B34T5MGSM3WGZN9A72 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:38.820" grpc.time_ms=0.405 pid=910 span.kind=server system=grpc1992gitlab # [ 253.409324] gitaly[910]: time="2026-11-10T10:58:38.836Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72HR8B34T5MGSM3WGZN9A72 diskcache=29844ce1-c340-44b5-b54f-de44af7c6b89 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T10:59:08.673" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:38.674" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1993gitlab # [ 253.418225] gitaly[910]: time="2026-11-10T10:58:38.844Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=74 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7382 command.oublock=0 command.real_time_ms=155 command.spawn_token_fork_ms=2 command.system_time_ms=50 command.user_time_ms=24 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR8B34T5MGSM3WGZN9A72 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T10:59:08.673" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:38.674" grpc.time_ms=162.005 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1994gitlab # [ 253.443493] nix-daemon[2832]: accepted connection from pid 3677, user nixbot1995gitlab # [ 253.460338] gitaly[910]: time="2026-11-10T10:58:38.886Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR8B34T5MGSM3WGZN9A72 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T10:59:08.885" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:38.885" grpc.time_ms=0.599 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1996gitlab # [ 253.480264] gitaly[910]: time="2026-11-10T10:58:38.906Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR8B34T5MGSM3WGZN9A72 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:08.006" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:38.906" grpc.time_ms=0.462 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1997gitlab # [ 253.486136] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:38.912Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"fcee583f88a2b7e76855764b","created_at":"2026-11-10T10:58:37.651Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-6aff6339aa2f4a7a","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:37.654Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2545,"job_status":"rescheduled","message":"PipelineProcessWorker JID-fcee583f88a2b7e76855764b: rescheduled"}1998gitlab # [ 253.490899] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:38.917Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"fcee583f88a2b7e76855764b","created_at":"2026-11-10T10:58:37.651Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-6aff6339aa2f4a7a","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:37.654Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fcee583f88a2b7e76855764b: done: 0.321886 sec","job_status":"done","queue_duration_s":0.940933,"scheduling_latency_s":0.940952,"gitaly_calls":3,"gitaly_duration_s":0.168917,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.022076,"redis_read_bytes":151,"redis_write_bytes":18014,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000914,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000622,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.001971,"redis_queues_read_bytes":29,"redis_queues_write_bytes":12313,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.003998,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000186,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000653,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.012818,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.023,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.023,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.089824,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.286826678,"duration_s":0.321886,"completed_at":"2026-11-10T10:58:38.917Z","load_balancing_strategy":"primary","db_duration_s":0.041674,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1999gitlab # [ 253.510472] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:38.918Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","ab4a35ac5f2728563965b525"],"class":"WebHooks::LogExecutionWorker","jid":"72b3e9e299ff2057642a4043","created_at":"2026-11-10T10:58:38.241Z","trace_propagation_headers":{"sentry-trace":"2bdc67a45a86414dbe01f8a2fef87a19-df3eba6e63cb419c","baggage":"sentry-trace_id=2bdc67a45a86414dbe01f8a2fef87a19,sentry-release=Unknown"},"correlation_id":"01M72HR3R3MA3WDV8JDVCXNTVF","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:490f5f356d0fdda47215af81b6a3360511eb7ac2ddad2774cae38de5ee499d11","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.244Z","job_size_bytes":2591,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-72b3e9e299ff2057642a4043: start","job_status":"start","queue_duration_s":0.67372,"scheduling_latency_s":0.673737}2000gitlab # [ 253.593404] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.019Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","ab4a35ac5f2728563965b525"],"class":"WebHooks::LogExecutionWorker","jid":"72b3e9e299ff2057642a4043","created_at":"2026-11-10T10:58:38.241Z","trace_propagation_headers":{"sentry-trace":"2bdc67a45a86414dbe01f8a2fef87a19-df3eba6e63cb419c","baggage":"sentry-trace_id=2bdc67a45a86414dbe01f8a2fef87a19,sentry-release=Unknown"},"correlation_id":"01M72HR3R3MA3WDV8JDVCXNTVF","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:490f5f356d0fdda47215af81b6a3360511eb7ac2ddad2774cae38de5ee499d11","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.244Z","job_size_bytes":2591,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-72b3e9e299ff2057642a4043: done: 0.100599 sec","job_status":"done","queue_duration_s":0.67372,"scheduling_latency_s":0.673737,"redis_calls":7,"redis_duration_s":0.002094,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000465,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.068,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.068,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.100599,"completed_at":"2026-11-10T10:58:39.019Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.070181,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2001gitlab # [ 253.634523] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.060Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"7a94277126954e93d3cfdeb2","created_at":"2026-11-10T10:58:38.274Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-3d449b7e6bdd44df","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.281Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-7a94277126954e93d3cfdeb2: start","job_status":"start","queue_duration_s":0.778936,"scheduling_latency_s":0.778985}2002gitlab # [ 253.740278] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.165Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"7a94277126954e93d3cfdeb2","created_at":"2026-11-10T10:58:38.274Z","trace_propagation_headers":{"sentry-trace":"c9ad1569a2584bbaa61285fc08695985-3d449b7e6bdd44df","baggage":"sentry-trace_id=c9ad1569a2584bbaa61285fc08695985,sentry-release=Unknown"},"correlation_id":"01M72HPNHC58J4F129DPYHBM88","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.281Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-7a94277126954e93d3cfdeb2: done: 0.105771 sec","job_status":"done","queue_duration_s":0.778936,"scheduling_latency_s":0.778985,"redis_calls":11,"redis_duration_s":0.003175,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000683,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001916,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000576,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000297524,"duration_s":0.105771,"completed_at":"2026-11-10T10:58:39.165Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024502,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2003gitlab # [ 253.742411] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.167Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"4625f8fa68da81d6db889ed5","created_at":"2026-11-10T10:58:38.622Z","trace_propagation_headers":{"sentry-trace":"d0008a36f92846348dd28d5f8883ffba-02dbd03032ca452f","baggage":"sentry-trace_id=d0008a36f92846348dd28d5f8883ffba,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HR97SB98R6MWKRW8R2XMY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.625Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-4625f8fa68da81d6db889ed5: start","job_status":"start","queue_duration_s":0.541735,"scheduling_latency_s":0.541751}2004gitlab # [ 253.814393] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.190Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"4625f8fa68da81d6db889ed5","created_at":"2026-11-10T10:58:38.622Z","trace_propagation_headers":{"sentry-trace":"d0008a36f92846348dd28d5f8883ffba-02dbd03032ca452f","baggage":"sentry-trace_id=d0008a36f92846348dd28d5f8883ffba,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HR97SB98R6MWKRW8R2XMY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.625Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-4625f8fa68da81d6db889ed5: done: 0.023073 sec","job_status":"done","queue_duration_s":0.541735,"scheduling_latency_s":0.541751,"redis_calls":7,"redis_duration_s":0.001717,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001248,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000469,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023073,"completed_at":"2026-11-10T10:58:39.190Z","load_balancing_strategy":"primary","db_duration_s":0.006089,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2005gitlab # [ 253.825664] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.191Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"ded52f7e84a4cd63f61b6050","created_at":"2026-11-10T10:58:38.627Z","trace_propagation_headers":{"sentry-trace":"d0008a36f92846348dd28d5f8883ffba-02dbd03032ca452f","baggage":"sentry-trace_id=d0008a36f92846348dd28d5f8883ffba,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HR97SB98R6MWKRW8R2XMY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.631Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-ded52f7e84a4cd63f61b6050: start","job_status":"start","queue_duration_s":0.560651,"scheduling_latency_s":0.560667}2006gitlab # [ 253.893603] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.319Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"ded52f7e84a4cd63f61b6050","created_at":"2026-11-10T10:58:38.627Z","trace_propagation_headers":{"sentry-trace":"d0008a36f92846348dd28d5f8883ffba-02dbd03032ca452f","baggage":"sentry-trace_id=d0008a36f92846348dd28d5f8883ffba,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HR97SB98R6MWKRW8R2XMY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.631Z","job_size_bytes":65,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-ded52f7e84a4cd63f61b6050: done: 0.128259 sec","job_status":"done","queue_duration_s":0.560651,"scheduling_latency_s":0.560667,"redis_calls":10,"redis_duration_s":0.0024879999999999998,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001142,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001346,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.128259,"completed_at":"2026-11-10T10:58:39.319Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.100361,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2007gitlab # [ 253.895065] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.321Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"79ca3ff8ed1e14273fea1f29","created_at":"2026-11-10T10:58:38.668Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.669Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-79ca3ff8ed1e14273fea1f29: start","job_status":"start","queue_duration_s":0.651613,"scheduling_latency_s":0.65163}2008gitlab # [ 253.939122] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.352Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"79ca3ff8ed1e14273fea1f29","created_at":"2026-11-10T10:58:38.668Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.669Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-79ca3ff8ed1e14273fea1f29: done: 0.031054 sec","job_status":"done","queue_duration_s":0.651613,"scheduling_latency_s":0.65163,"redis_calls":7,"redis_duration_s":0.001667,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001116,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000551,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031054,"completed_at":"2026-11-10T10:58:39.352Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008611,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2009gitlab # [ 253.947707] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.353Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"7702a18f37b78891815ed620","created_at":"2026-11-10T10:58:38.671Z","trace_propagation_headers":{"sentry-trace":"d0008a36f92846348dd28d5f8883ffba-02dbd03032ca452f","baggage":"sentry-trace_id=d0008a36f92846348dd28d5f8883ffba,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HR97SB98R6MWKRW8R2XMY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.673Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7702a18f37b78891815ed620: start","job_status":"start","queue_duration_s":0.680249,"scheduling_latency_s":0.680265}2010gitlab # [ 253.985795] gitaly[910]: time="2026-11-10T10:58:39.412Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HR97SB98R6MWKRW8R2XMY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T10:59:09.410" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:39.411" grpc.time_ms=1.01 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2011gitlab # [ 254.035430] gitaly[910]: time="2026-11-10T10:58:39.462Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HR97SB98R6MWKRW8R2XMY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:09.460" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:39.461" grpc.time_ms=0.692 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2012gitlab # [ 254.046675] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.472Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"7702a18f37b78891815ed620","created_at":"2026-11-10T10:58:38.671Z","trace_propagation_headers":{"sentry-trace":"d0008a36f92846348dd28d5f8883ffba-02dbd03032ca452f","baggage":"sentry-trace_id=d0008a36f92846348dd28d5f8883ffba,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HR97SB98R6MWKRW8R2XMY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.673Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7702a18f37b78891815ed620: done: 0.118789 sec","job_status":"done","queue_duration_s":0.680249,"scheduling_latency_s":0.680265,"gitaly_calls":2,"gitaly_duration_s":0.007777,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.007785,"redis_read_bytes":80,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001028,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00182,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000305,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001652,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000238,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000648,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001066,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081885,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.094207225,"duration_s":0.118789,"completed_at":"2026-11-10T10:58:39.472Z","load_balancing_strategy":"primary","db_duration_s":0.027356,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2013gitlab # [ 254.053708] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.474Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"17510dfb0912fea932f15bbc","created_at":"2026-11-10T10:58:38.840Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.842Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-17510dfb0912fea932f15bbc: start","job_status":"start","queue_duration_s":0.632015,"scheduling_latency_s":0.632032}2014gitlab # [ 254.081245] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.507Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"17510dfb0912fea932f15bbc","created_at":"2026-11-10T10:58:38.840Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.842Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-17510dfb0912fea932f15bbc: done: 0.032857 sec","job_status":"done","queue_duration_s":0.632015,"scheduling_latency_s":0.632032,"redis_calls":7,"redis_duration_s":0.002225,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001573,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000652,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032857,"completed_at":"2026-11-10T10:58:39.507Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004641,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2015gitlab # [ 254.136738] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.509Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f24824ed40555e6e685d82c7","created_at":"2026-11-10T10:58:38.860Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.861Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f24824ed40555e6e685d82c7: start","job_status":"start","queue_duration_s":0.648087,"scheduling_latency_s":0.648121}2016gitlab # [ 254.137547] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.546Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f24824ed40555e6e685d82c7","created_at":"2026-11-10T10:58:38.860Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.861Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f24824ed40555e6e685d82c7: done: 0.036469 sec","job_status":"done","queue_duration_s":0.648087,"scheduling_latency_s":0.648121,"redis_calls":7,"redis_duration_s":0.002228,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001563,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000665,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036469,"completed_at":"2026-11-10T10:58:39.546Z","load_balancing_strategy":"primary","db_duration_s":0.006504,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2017gitlab # [ 254.180253] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.548Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"672fff1cfb92d632b811262a","created_at":"2026-11-10T10:58:38.867Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.868Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-672fff1cfb92d632b811262a: start","job_status":"start","queue_duration_s":0.679962,"scheduling_latency_s":0.679997}2018gitlab # [ 254.180805] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.573Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"672fff1cfb92d632b811262a","created_at":"2026-11-10T10:58:38.867Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.868Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-672fff1cfb92d632b811262a: done: 0.024668 sec","job_status":"done","queue_duration_s":0.679962,"scheduling_latency_s":0.679997,"redis_calls":10,"redis_duration_s":0.002303,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001172,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000663,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000468,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024668,"completed_at":"2026-11-10T10:58:39.573Z","load_balancing_strategy":"primary","db_duration_s":0.004536,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2019gitlab # [ 254.220074] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.574Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"07ce80af69080bd40cb352a6","created_at":"2026-11-10T10:58:38.868Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.870Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-07ce80af69080bd40cb352a6: start","job_status":"start","queue_duration_s":0.704219,"scheduling_latency_s":0.704237}2020gitlab # [ 254.262300] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.594Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"07ce80af69080bd40cb352a6","created_at":"2026-11-10T10:58:38.868Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.870Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-07ce80af69080bd40cb352a6: done: 0.020127 sec","job_status":"done","queue_duration_s":0.704219,"scheduling_latency_s":0.704237,"redis_calls":7,"redis_duration_s":0.001633,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0012,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000433,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020127,"completed_at":"2026-11-10T10:58:39.594Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004938,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2021gitlab # [ 254.300764] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.595Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"51378b4d900a02ef7ecbe289","created_at":"2026-11-10T10:58:38.870Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.871Z","job_size_bytes":19,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-51378b4d900a02ef7ecbe289: start","job_status":"start","queue_duration_s":0.723878,"scheduling_latency_s":0.723893}2022gitlab # [ 254.301530] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.615Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"51378b4d900a02ef7ecbe289","created_at":"2026-11-10T10:58:38.870Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.871Z","job_size_bytes":19,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-51378b4d900a02ef7ecbe289: done: 0.019503 sec","job_status":"done","queue_duration_s":0.723878,"scheduling_latency_s":0.723893,"redis_calls":8,"redis_duration_s":0.001902,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001431,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000471,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019503,"completed_at":"2026-11-10T10:58:39.615Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005088,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2023gitlab # [ 254.344554] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.616Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"6cfc989d77069a1d3f564cb4","created_at":"2026-11-10T10:58:38.872Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.873Z","job_size_bytes":66,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-6cfc989d77069a1d3f564cb4: start","job_status":"start","queue_duration_s":0.742921,"scheduling_latency_s":0.742946}2024gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses2025gitlab # [ 254.345233] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.633Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"6cfc989d77069a1d3f564cb4","created_at":"2026-11-10T10:58:38.872Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.873Z","job_size_bytes":66,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-6cfc989d77069a1d3f564cb4: done: 0.016747 sec","job_status":"done","queue_duration_s":0.742921,"scheduling_latency_s":0.742946,"redis_calls":7,"redis_duration_s":0.001628,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001186,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010029,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016747,"completed_at":"2026-11-10T10:58:39.633Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003199,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2026gitlab # [ 254.384620] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.634Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"f4b3555c885772f0b732b190","created_at":"2026-11-10T10:58:38.873Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.874Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-f4b3555c885772f0b732b190: start","job_status":"start","queue_duration_s":0.759768,"scheduling_latency_s":0.759784}2027gitlab # [ 254.385239] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.657Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"f4b3555c885772f0b732b190","created_at":"2026-11-10T10:58:38.873Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.874Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-f4b3555c885772f0b732b190: done: 0.022511 sec","job_status":"done","queue_duration_s":0.759768,"scheduling_latency_s":0.759784,"redis_calls":7,"redis_duration_s":0.001232,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000823,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000409,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022511,"completed_at":"2026-11-10T10:58:39.657Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0084,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2028gitlab # [ 254.426191] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.658Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"b01ffa866bc2058187ee08ec","created_at":"2026-11-10T10:58:38.875Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.876Z","job_size_bytes":6,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-b01ffa866bc2058187ee08ec: start","job_status":"start","queue_duration_s":0.782222,"scheduling_latency_s":0.78224}2029gitlab # [ 254.434442] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.668Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"b01ffa866bc2058187ee08ec","created_at":"2026-11-10T10:58:38.875Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.876Z","job_size_bytes":6,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-b01ffa866bc2058187ee08ec: done: 0.009647 sec","job_status":"done","queue_duration_s":0.782222,"scheduling_latency_s":0.78224,"redis_calls":7,"redis_duration_s":0.0008150000000000001,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000591,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000224,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009647,"completed_at":"2026-11-10T10:58:39.668Z","load_balancing_strategy":"primary","db_duration_s":0.000413,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2030gitlab # [ 254.480111] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.669Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"bf5a0cff85efffdc8e0409f6","created_at":"2026-11-10T10:58:38.876Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.877Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-bf5a0cff85efffdc8e0409f6: start","job_status":"start","queue_duration_s":0.791515,"scheduling_latency_s":0.791528}2031gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses, in 0.15 seconds)2032gitlab # [ 254.485266] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.685Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"bf5a0cff85efffdc8e0409f6","created_at":"2026-11-10T10:58:38.876Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.877Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-bf5a0cff85efffdc8e0409f6: done: 0.016403 sec","job_status":"done","queue_duration_s":0.791515,"scheduling_latency_s":0.791528,"redis_calls":7,"redis_duration_s":0.001086,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010236,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016403,"completed_at":"2026-11-10T10:58:39.685Z","load_balancing_strategy":"primary","db_duration_s":0.003686,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2033gitlab # [ 254.539155] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.686Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"36868a258c4709c66ac08e81","created_at":"2026-11-10T10:58:38.912Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.914Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-36868a258c4709c66ac08e81: start","job_status":"start","queue_duration_s":0.772444,"scheduling_latency_s":0.772458}2034gitlab # [ 254.539762] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.705Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"36868a258c4709c66ac08e81","created_at":"2026-11-10T10:58:38.912Z","trace_propagation_headers":{"sentry-trace":"5fa4012cafa34220b4c5a7647fee5c48-9478916b5c6847db","baggage":"sentry-trace_id=5fa4012cafa34220b4c5a7647fee5c48,sentry-release=Unknown"},"correlation_id":"01M72HR8B34T5MGSM3WGZN9A72","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:38.914Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-36868a258c4709c66ac08e81: done: 0.019366 sec","job_status":"done","queue_duration_s":0.772444,"scheduling_latency_s":0.772458,"redis_calls":8,"redis_duration_s":0.002051,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001481,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00057,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019366,"completed_at":"2026-11-10T10:58:39.705Z","load_balancing_strategy":"primary","db_duration_s":0.004039,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2035gitlab # [ 254.573465] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 942036gitlab # [ 254.575790] gitaly[910]: time="2026-11-10T10:58:39.913Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HRAKK270Q0DDPRNBKWVWA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:09.511" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:39.912" grpc.time_ms=0.84 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2037gitlab # [ 254.586774] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.707Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"96fed43192713b62252b692b","created_at":"2026-11-10T10:58:39.398Z","trace_propagation_headers":{"sentry-trace":"d0008a36f92846348dd28d5f8883ffba-57fdf96246c648a5","baggage":"sentry-trace_id=d0008a36f92846348dd28d5f8883ffba,sentry-release=Unknown"},"correlation_id":"01M72HR97SB98R6MWKRW8R2XMY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:39.399Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-96fed43192713b62252b692b: start","job_status":"start","queue_duration_s":0.307341,"scheduling_latency_s":0.307356}2038gitlab # [ 254.601209] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:39.732Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"96fed43192713b62252b692b","created_at":"2026-11-10T10:58:39.398Z","trace_propagation_headers":{"sentry-trace":"d0008a36f92846348dd28d5f8883ffba-57fdf96246c648a5","baggage":"sentry-trace_id=d0008a36f92846348dd28d5f8883ffba,sentry-release=Unknown"},"correlation_id":"01M72HR97SB98R6MWKRW8R2XMY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:39.399Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-96fed43192713b62252b692b: done: 0.025455 sec","job_status":"done","queue_duration_s":0.307341,"scheduling_latency_s":0.307356,"redis_calls":7,"redis_duration_s":0.002264,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001217,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025455,"completed_at":"2026-11-10T10:58:39.732Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00516,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2039gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses2040gitlab # [ 255.632573] gitaly[910]: time="2026-11-10T10:58:41.058Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HRBQ4HNMXWE9S61HTG788 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:11.056" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:41.057" grpc.time_ms=0.897 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2041gitlab # [ 255.674696] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:41 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1042042gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses, in 0.15 seconds)2043gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses2044gitlab # [ 256.766598] gitaly[910]: time="2026-11-10T10:58:42.192Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HRCTTMCSNPZYYFM7PE73J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:12.190" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:42.191" grpc.time_ms=0.855 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2045gitlab # [ 256.803226] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:42 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 912046gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses, in 0.13 seconds)2047gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses2048gitlab # [ 257.879549] nix-daemon[2832]: accepted connection from pid 3710, user nixbot2049gitlab # [ 257.892695] gitaly[910]: time="2026-11-10T10:58:43.318Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HRDY56HFJ2XBYWZ8WJ2BC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:13.317" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:43.317" grpc.time_ms=0.868 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2050gitlab # [ 257.911550] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 682051gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses, in 0.12 seconds)2052gitlab # [ 257.928642] nix-daemon[2832]: accepted connection from pid 3677, user nixbot2053gitlab # [ 258.012721] gitaly[910]: time="2026-11-10T10:58:43.439Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:13.437" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:43.438" grpc.time_ms=0.776 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2054gitlab # [ 258.091455] gitaly[910]: time="2026-11-10T10:58:43.518Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:53.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:43.508" grpc.time_ms=9.671 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2055gitlab # [ 258.163574] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:43.588Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8a2738983b5b39baa1466496","created_at":"2026-11-10T10:58:43.583Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-2bdeae9090ab41bb","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.586Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8a2738983b5b39baa1466496: start","job_status":"start","queue_duration_s":0.001724,"scheduling_latency_s":0.001769}2056gitlab # [ 258.214846] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2522057gitlab # [ 258.272664] gitaly[910]: time="2026-11-10T10:58:43.699Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HREA4S2VHSXAYXAHA5JMT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:13.698" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:43.698" grpc.time_ms=0.644 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2058gitlab # [ 258.303994] gitaly[910]: time="2026-11-10T10:58:43.730Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M72HREA4S2VHSXAYXAHA5JMT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:53.224" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:43.724" grpc.time_ms=6.164 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2059gitlab # [ 258.333653] gitaly[910]: time="2026-11-10T10:58:43.760Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:43.760" grpc.time_ms=0.284 pid=910 span.kind=server system=grpc2060gitlab # [ 258.338249] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1122061gitlab # [ 258.365282] gitaly[910]: time="2026-11-10T10:58:43.792Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:43.791" grpc.time_ms=0.646 pid=910 span.kind=server system=grpc2062gitlab # [ 258.389462] gitaly[910]: time="2026-11-10T10:58:43.816Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HREE3VD8C8TZFFE85VTTG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:13.815" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:43.815" grpc.time_ms=0.474 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2063gitlab # [ 258.402387] gitaly[910]: time="2026-11-10T10:58:43.829Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:43.828" grpc.time_ms=0.406 pid=910 span.kind=server system=grpc2064gitlab # [ 258.414583] gitaly[910]: time="2026-11-10T10:58:43.840Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T10:59:13.740" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:43.740" operations="[repack_refs]" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2065gitlab # [ 258.421041] gitaly[910]: time="2026-11-10T10:58:43.847Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X diskcache=e1c9736f-6646-4e91-8749-54227728001f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T10:59:13.740" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:43.740" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2066gitlab # [ 258.421726] gitaly[910]: time="2026-11-10T10:58:43.847Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=90 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7275 command.oublock=0 command.real_time_ms=99 command.spawn_token_fork_ms=1 command.system_time_ms=69 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T10:59:13.740" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:43.740" grpc.time_ms=107.208 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2067gitlab # [ 258.424422] gitaly[910]: time="2026-11-10T10:58:43.851Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HREE3VD8C8TZFFE85VTTG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-11-10T10:58:53.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:43.846" grpc.time_ms=4.839 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2068gitlab # [ 258.458310] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/11/10:10:58:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1052069gitlab # [ 258.458664] gitaly[910]: time="2026-11-10T10:58:43.885Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T10:59:13.740" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:43.740" pid=910 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-11-10 10:57:38.75499803 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2070gitlab # [ 258.467219] gitaly[910]: time="2026-11-10T10:58:43.893Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-11-10T10:59:13.893" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:43.893" grpc.time_ms=0.51 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2071gitlab # [ 258.495404] gitaly[910]: time="2026-11-10T10:58:43.922Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-11-10T10:59:13.020" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-11-10T10:58:43.920" grpc.time_ms=1.049 pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2072gitlab # [ 258.507450] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:43.931Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f5dbfe336f6aa7666ac0093a","created_at":"2026-11-10T10:58:43.929Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"8a2738983b5b39baa1466496","job_size_bytes":3,"pid":2545,"job_status":"deduplicated","message":"PipelineProcessWorker JID-f5dbfe336f6aa7666ac0093a: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2073gitlab # [ 258.507802] gitaly[910]: time="2026-11-10T10:58:43.934Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T10:59:13.740" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:43.740" pid=910 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2074gitlab # [ 258.508401] gitaly[910]: time="2026-11-10T10:58:43.934Z" level=info msg="optimized repository" clean_stale_data_time=48.730774ms clean_work_trees_time="84.648µs" commit_graph_time="3.911µs" correlation_id=01M72HRE1XQZEJK2ZXVEEF595X gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="3.911µs" pid=910 prune_time="3.352µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="4.19µs" storage_name=default system=housekeeping total_time=48.92214ms2075gitlab # [ 258.508962] gitaly[910]: time="2026-11-10T10:58:43.934Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T10:59:13.740" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:43.740" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2076gitlab # [ 258.509570] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:43.934Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8a2738983b5b39baa1466496","created_at":"2026-11-10T10:58:43.583Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-2bdeae9090ab41bb","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.586Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2545,"job_status":"rescheduled","message":"PipelineProcessWorker JID-8a2738983b5b39baa1466496: rescheduled"}2077gitlab # [ 258.520598] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:43.939Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8a2738983b5b39baa1466496","created_at":"2026-11-10T10:58:43.583Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-2bdeae9090ab41bb","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.586Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8a2738983b5b39baa1466496: done: 0.3509 sec","job_status":"done","queue_duration_s":0.001724,"scheduling_latency_s":0.001769,"gitaly_calls":3,"gitaly_duration_s":0.116693,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.0141,"redis_read_bytes":239,"redis_write_bytes":21782,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001321,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001058,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.002328,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14467,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.005948,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000343,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000348,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001433,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.012,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.132811,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.269746016,"duration_s":0.3509,"completed_at":"2026-11-10T10:58:43.939Z","load_balancing_strategy":"primary","db_duration_s":0.057059,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2078gitlab # [ 258.593727] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:43.940Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d4e0e5b0b599a2f1cd9fba73","created_at":"2026-11-10T10:58:43.731Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.734Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d4e0e5b0b599a2f1cd9fba73: start","job_status":"start","queue_duration_s":0.20634,"scheduling_latency_s":0.206359}2079gitlab # [ 258.594622] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:43.967Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"d4e0e5b0b599a2f1cd9fba73","created_at":"2026-11-10T10:58:43.731Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.734Z","job_size_bytes":23,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-d4e0e5b0b599a2f1cd9fba73: done: 0.027125 sec","job_status":"done","queue_duration_s":0.20634,"scheduling_latency_s":0.206359,"redis_calls":7,"redis_duration_s":0.000907,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000652,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027125,"completed_at":"2026-11-10T10:58:43.967Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004607,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2080gitlab # [ 258.645999] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:43.968Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9e3f22ce057abd9cbc4bbd9e","created_at":"2026-11-10T10:58:43.854Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.856Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9e3f22ce057abd9cbc4bbd9e: start","job_status":"start","queue_duration_s":0.112218,"scheduling_latency_s":0.112234}2081gitlab # [ 258.646737] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:43.987Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9e3f22ce057abd9cbc4bbd9e","created_at":"2026-11-10T10:58:43.854Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.856Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9e3f22ce057abd9cbc4bbd9e: done: 0.01869 sec","job_status":"done","queue_duration_s":0.112218,"scheduling_latency_s":0.112234,"redis_calls":7,"redis_duration_s":0.000826,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000611,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000215,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012421,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01869,"completed_at":"2026-11-10T10:58:43.987Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003747,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2082gitlab # [ 258.693722] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:43.988Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"475f5f86283f6ecb8c7504d9","created_at":"2026-11-10T10:58:43.861Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.862Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-475f5f86283f6ecb8c7504d9: start","job_status":"start","queue_duration_s":0.125121,"scheduling_latency_s":0.125137}2083gitlab # [ 258.694531] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:44.005Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"475f5f86283f6ecb8c7504d9","created_at":"2026-11-10T10:58:43.861Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.862Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-475f5f86283f6ecb8c7504d9: done: 0.017307 sec","job_status":"done","queue_duration_s":0.125121,"scheduling_latency_s":0.125137,"redis_calls":7,"redis_duration_s":0.000846,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000612,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000234,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017307,"completed_at":"2026-11-10T10:58:44.005Z","load_balancing_strategy":"primary","db_duration_s":0.00278,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2084gitlab # [ 258.747529] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:44.006Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"fbdce8afe77079634b4c0187","created_at":"2026-11-10T10:58:43.868Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.869Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fbdce8afe77079634b4c0187: start","job_status":"start","queue_duration_s":0.136888,"scheduling_latency_s":0.136903}2085gitlab # [ 258.748481] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:44.026Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"fbdce8afe77079634b4c0187","created_at":"2026-11-10T10:58:43.868Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.869Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fbdce8afe77079634b4c0187: done: 0.020352 sec","job_status":"done","queue_duration_s":0.136888,"scheduling_latency_s":0.136903,"redis_calls":10,"redis_duration_s":0.001184,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000605,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000362,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000217,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020352,"completed_at":"2026-11-10T10:58:44.026Z","load_balancing_strategy":"primary","db_duration_s":0.003032,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2086gitlab # [ 258.794160] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:44.028Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6f101734ddaaf298371bebeb","created_at":"2026-11-10T10:58:43.869Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.870Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6f101734ddaaf298371bebeb: start","job_status":"start","queue_duration_s":0.157138,"scheduling_latency_s":0.157154}2087gitlab # [ 258.794831] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:44.045Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"6f101734ddaaf298371bebeb","created_at":"2026-11-10T10:58:43.869Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.870Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-6f101734ddaaf298371bebeb: done: 0.017556 sec","job_status":"done","queue_duration_s":0.157138,"scheduling_latency_s":0.157154,"redis_calls":7,"redis_duration_s":0.000879,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000652,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000227,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017556,"completed_at":"2026-11-10T10:58:44.045Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002929,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2088gitlab # [ 258.842223] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:44.046Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d2bfb9f38e48bcb4c065f418","created_at":"2026-11-10T10:58:43.871Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.873Z","job_size_bytes":19,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d2bfb9f38e48bcb4c065f418: start","job_status":"start","queue_duration_s":0.173035,"scheduling_latency_s":0.173058}2089gitlab # [ 258.848927] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:44.061Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d2bfb9f38e48bcb4c065f418","created_at":"2026-11-10T10:58:43.871Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.873Z","job_size_bytes":19,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d2bfb9f38e48bcb4c065f418: done: 0.015187 sec","job_status":"done","queue_duration_s":0.173035,"scheduling_latency_s":0.173058,"redis_calls":8,"redis_duration_s":0.001286,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001074,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000212,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009843,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015187,"completed_at":"2026-11-10T10:58:44.061Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002155,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2090gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses2091gitlab # [ 258.891089] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:44.062Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"f9dd6a2233462f4946132ba4","created_at":"2026-11-10T10:58:43.874Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.875Z","job_size_bytes":66,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-f9dd6a2233462f4946132ba4: start","job_status":"start","queue_duration_s":0.186691,"scheduling_latency_s":0.186708}2092gitlab # [ 258.928952] gitaly[910]: time="2026-11-10T10:58:44.222Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:44.222" grpc.time_ms=0.266 pid=910 span.kind=server system=grpc2093gitlab # [ 258.929642] gitaly[910]: time="2026-11-10T10:58:44.242Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:44.241" grpc.time_ms=0.498 pid=910 span.kind=server system=grpc2094gitlab # [ 258.930233] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:44.076Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"f9dd6a2233462f4946132ba4","created_at":"2026-11-10T10:58:43.874Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.875Z","job_size_bytes":66,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-f9dd6a2233462f4946132ba4: done: 0.01398 sec","job_status":"done","queue_duration_s":0.186691,"scheduling_latency_s":0.186708,"redis_calls":7,"redis_duration_s":0.0008179999999999999,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000217,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009464,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01398,"completed_at":"2026-11-10T10:58:44.076Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001842,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2095gitlab # [ 258.979690] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:44.077Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"af84cf9c071d9951a08b6505","created_at":"2026-11-10T10:58:43.876Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.877Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-af84cf9c071d9951a08b6505: start","job_status":"start","queue_duration_s":0.199884,"scheduling_latency_s":0.1999}2096gitlab # [ 258.980633] gitaly[910]: time="2026-11-10T10:58:44.259Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-11-10T10:58:44.259" grpc.time_ms=0.24 pid=910 span.kind=server system=grpc2097gitlab # [ 258.981369] gitaly[910]: time="2026-11-10T10:58:44.268Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X diskcache=924ee7d2-087f-4209-92c7-a7ce37b16c71 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T10:59:14.201" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-11-10T10:58:44.201" pid=910 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2098gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1/statuses, in 0.14 seconds)2099gitlab # [ 258.986653] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:44.098Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"af84cf9c071d9951a08b6505","created_at":"2026-11-10T10:58:43.876Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.877Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-af84cf9c071d9951a08b6505: done: 0.021186 sec","job_status":"done","queue_duration_s":0.199884,"scheduling_latency_s":0.1999,"redis_calls":7,"redis_duration_s":0.000847,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000218,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021186,"completed_at":"2026-11-10T10:58:44.098Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002929,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2100(finished: subtest: eval and build post commit statuses, in 5.80 seconds)2101(finished: run the VM test script, in 320.09 seconds)2102gitlab # [ 259.035711] sidekiq-cluster[2545]: {"severity":"INFO","time":"2026-11-10T10:58:44.099Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"c129a47bfcd80706b0e4295e","created_at":"2026-11-10T10:58:43.880Z","trace_propagation_headers":{"sentry-trace":"5349e0976011475585ad550516c28aaa-752561dcb4f44188","baggage":"sentry-trace_id=5349e0976011475585ad550516c28aaa,sentry-release=Unknown"},"correlation_id":"01M72HRE1XQZEJK2ZXVEEF595X","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-11-10T10:58:43.881Z","job_size_bytes":3,"pid":2545,"sidekiq_tid":"m7xh","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-c129a47bfcd80706b0e4295e: start","job_status":"start","queue_duration_s":0.217922,"scheduling_latency_s":0.21794}2103test script finished in 320.11s2104cleanup2105kill QemuMachine (pid 46)2106gitlab # [ 259.038972] gitaly[910]: time="2026-11-10T10:58:44.268Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=65 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7494 command.oublock=0 command.real_time_ms=60 command.spawn_token_fork_ms=0 command.system_time_ms=44 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M72HRE1XQZEJK2ZXVEEF595X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-11-10T10:59:14.201" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/tesqemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14)2107gitlab # t2108(finished: cleanup, in 1.52 seconds)2109additionally exposed symbols:2110 gitlab,2111 vlan1,2112 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2113[]2114[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-11-10T10:57:49.099Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2115[]2116[]2117[]2118[]2119[]2120[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-11-10T10:58:38.451811Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}]2121[{'id': 5, 'sha': 'fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T10:58:38.667Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2122[{'id': 5, 'sha': 'fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T10:58:38.667Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2123[{'id': 5, 'sha': 'fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T10:58:38.667Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2124[{'id': 5, 'sha': 'fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T10:58:38.667Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2125[{'id': 5, 'sha': 'fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-11-10T10:58:38.667Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2126[{'id': 5, 'sha': 'fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 4s (1 warning)', 'created_at': '2026-11-10T10:58:38.667Z', 'started_at': None, 'finished_at': '2026-11-10T10:58:43.572Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'fa2bae0b1e46cc6ff8d4932a45a382ce04467ff1', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-11-10T10:58:43.746Z', 'started_at': None, 'finished_at': '2026-11-10T10:58:43.865Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]